doc strings

42 views
Skip to first unread message

Arthur A. Gleckler

unread,
Apr 5, 2020, 12:49:30 AM4/5/20
to scheme-re...@googlegroups.com, John Cowan
John, do you think that doc strings might be considered for R7RS Large? I've been including them in my code for years in anticipation of the day that they would actually be supported, and it occurs to me that now might be the time to speak up:

(define (find-previous-tail predicate list)
  "If the car of any pair of `list' after the first one satisfies
`predicate', return the previous pair.  Otherwise, return #f."
  (if (null? list)
      #f
      (let next ((previous list)
                 (tail (cdr list)))
        (cond ((null? tail) #f)
              ((predicate (car tail)) previous)
              (else (next (cdr previous)
                          (cdr tail)))))))

Marc Nieper-Wißkirchen

unread,
Apr 5, 2020, 3:34:21 AM4/5/20
to scheme-re...@googlegroups.com, John Cowan
May I chime in and ask about the semantics of the doc strings?

The following questions have come to my mind so far:

Are they supposed to be associated to objects at runtime or only at expand time? Or will they have no special meaning to the Scheme expander und runtime system but interpreted by a different tool as it is done with Scribble? What is the advantage of doc strings versus Scribble and vice versa?

How to associate doc strings to non-procedure definitions? How to syntax definitions? Do they work well with case-lambda? Shall doc strings only be associated to top-level definitions or to individual objects?

Scribble and some other documentation systems have some TeX-like markup language that allows to generate rich documentation and to extract meta-information. How are doc strings positioned in this regard?

Marc

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scheme-reports-wg2/CALnw4L%2Bh1JN7kdi45z_ULgcFwe0VorPUEQosvQqGvLX-nBBwFQ%40mail.gmail.com.

Arthur A. Gleckler

unread,
Apr 5, 2020, 3:45:37 AM4/5/20
to scheme-re...@googlegroups.com, John Cowan
On Sun, Apr 5, 2020 at 12:34 AM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
 
Are they supposed to be associated to objects at runtime or only at expand time? Or will they have no special meaning to the Scheme expander und runtime system but interpreted by a different tool as it is done with Scribble? What is the advantage of doc strings versus Scribble and vice versa?

Ideally, they would be associated with objects at runtime.  That doesn't mean that they would have to be kept in memory; only that some key for retrieving them would be present.  Ideally, a simple API could retrieve a doc string from any object that was defined with one.

Sorry, I don't yet know anything about Scribble.  I'll try to read up on it.
 
How to associate doc strings to non-procedure definitions?

I've seen systems that have a comment form that can be wrapped around any definition, attaching a string to the binding.
 
How to syntax definitions? Do they work well with case-lambda? Shall doc strings only be associated to top-level definitions or to individual objects?
 
Scribble and some other documentation systems have some TeX-like markup language that allows to generate rich documentation and to extract meta-information. How are doc strings positioned in this regard?

There's no reason that doc strings couldn't use markup as well, e.g. Markdown, but I'd be okay if that were left unspecified.

I would just like some simple way to attach a runtime-accessible string to each definition so that the REPL and other interactive tools can display it when requested.

Marc Nieper-Wißkirchen

unread,
Apr 5, 2020, 4:04:01 AM4/5/20
to scheme-re...@googlegroups.com, John Cowan
Am So., 5. Apr. 2020 um 09:45 Uhr schrieb Arthur A. Gleckler <a...@speechcode.com>:
On Sun, Apr 5, 2020 at 12:34 AM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
 
Are they supposed to be associated to objects at runtime or only at expand time? Or will they have no special meaning to the Scheme expander und runtime system but interpreted by a different tool as it is done with Scribble? What is the advantage of doc strings versus Scribble and vice versa?

Ideally, they would be associated with objects at runtime.  That doesn't mean that they would have to be kept in memory; only that some key for retrieving them would be present.  Ideally, a simple API could retrieve a doc string from any object that was defined with one.

Sorry, I don't yet know anything about Scribble.  I'll try to read up on it.

Chibi and Racket both use Scribble.  As far as I understand, Racket's (excellent) library documentation has been generated from it.

 
How to associate doc strings to non-procedure definitions?

I've seen systems that have a comment form that can be wrapped around any definition, attaching a string to the binding.

I would say such a comment form should rather wrap a value, not a definition as Scheme is very flexible when it comes to defining things:

(define-values (x y)
  (let ((foo (foobricator)))
    (values (comment "Twizzle foos one at a time" (make-proc foo 1)) (comment "Twizzle foos two at a time" (make-proc foo 2)))))

Doc strings in define forms would then just be syntactic sugar (Warning: Not implementable with syntax-rules!):

(define (name . formals) doc body) => (define name (comment doc (lambda formals body)))
(define name doc val) => (define name (comment doc val))
 
 
How to syntax definitions? Do they work well with case-lambda? Shall doc strings only be associated to top-level definitions or to individual objects?
 
Scribble and some other documentation systems have some TeX-like markup language that allows to generate rich documentation and to extract meta-information. How are doc strings positioned in this regard?

There's no reason that doc strings couldn't use markup as well, e.g. Markdown, but I'd be okay if that were left unspecified.

I would just like some simple way to attach a runtime-accessible string to each definition so that the REPL and other interactive tools can display it when requested.

In the case of syntactic bindings, we would attach this string to the actual transformer (which is, or can be seen, as a runtime object in a phase shifted by one)?

(define-syntax foo doc transformer) => (define-syntax foo (comment doc transformer))

Marc

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

Arne Babenhauserheide

unread,
Apr 5, 2020, 5:38:55 AM4/5/20
to scheme-re...@googlegroups.com, John Cowan

Marc Nieper-Wißkirchen <marc....@gmail.com> writes:

> Am So., 5. Apr. 2020 um 09:45 Uhr schrieb Arthur A. Gleckler <
> a...@speechcode.com>:
>
>> On Sun, Apr 5, 2020 at 12:34 AM Marc Nieper-Wißkirchen <
>> marc....@gmail.com> wrote:
>>
>>
>>> Are they supposed to be associated to objects at runtime or only at
>>> expand time? Or will they have no special meaning to the Scheme expander
>>> und runtime system but interpreted by a different tool as it is done with
>>> Scribble? What is the advantage of doc strings versus Scribble and vice
>>> versa?
>>>
>>
>> Ideally, they would be associated with objects at runtime. That doesn't
>> mean that they would have to be kept in memory; only that some key for
>> retrieving them would be present. Ideally, a simple API could retrieve a
>> doc string from any object that was defined with one.
>>
>> Sorry, I don't yet know anything about Scribble. I'll try to read up on
>> it.
>
> Chibi and Racket both use Scribble. As far as I understand, Racket's
> (excellent) library documentation has been generated from it.

Guile provides doc-strings as part of procedure-properties:

https://www.gnu.org/software/guile/docs/docs-2.0/guile-ref/Procedure-Properties.html#index-procedure_002ddocumentation


Minimal documented function:

(define (foo)
"doc"
#f)

(procedure-documentation foo) ; => "doc"
(foo) ; => #f


They are available at runtime.


To preserve Scheme-semantics,
(define (foo) "not-a-docstring")
has the return value "not-a-docstring", but no docstring.


In addition, procedures can have properties which can be used to
implement more natural doctests than for example seen in Python:

https://hg.sr.ht/~arnebab/wisp/browse/33b4fcdd8a17aa19d57971e4f6db5fcb7758843c/examples/doctests.scm#L10

Their advantage over Python-doctests is that test and implementation are
uniform: The test uses regular s-exp syntax instead of being
string-parsed as it is done in Python (with all the quoting hassles
string-parsing brings).

Best wishes,
Arne
--
Unpolitisch sein
heißt politisch sein
ohne es zu merken

Marc Nieper-Wißkirchen

unread,
Apr 5, 2020, 5:43:12 AM4/5/20
to scheme-re...@googlegroups.com, John Cowan
Is this feature of Guile well-thought out?

(1) There are more values than just procedures; we want to associate doc strings to, say, global constants or macro transformers as well.

(2) The convention that the first expression in a procedures body is a docstring if it is a string and not the only expression doesn't work with case-lambdas, because we don't want each individual clause to be documented but the whole procedure.

Marc



They are available at runtime.


To preserve Scheme-semantics,
(define (foo) "not-a-docstring")
has the return value "not-a-docstring", but no docstring.


In addition, procedures can have properties which can be used to
implement more natural doctests than for example seen in Python:

https://hg.sr.ht/~arnebab/wisp/browse/33b4fcdd8a17aa19d57971e4f6db5fcb7758843c/examples/doctests.scm#L10

Their advantage over Python-doctests is that test and implementation are
uniform: The test uses regular s-exp syntax instead of being
string-parsed as it is done in Python (with all the quoting hassles
string-parsing brings).

Best wishes,
Arne
--
Unpolitisch sein
heißt politisch sein
ohne es zu merken

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

Arthur A. Gleckler

unread,
Apr 5, 2020, 3:44:19 PM4/5/20
to scheme-re...@googlegroups.com, John Cowan

On Sun, Apr 5, 2020 at 12:45 AM Arthur A. Gleckler <a...@speechcode.com> wrote:

Sorry, I don't yet know anything about Scribble. I'll try to read up on it.

Okay, I found 4.5 In-Source Documentation and the In-Source Documentation section of Submodules. The latter gives examples. It is geared to support Racket's contract system, but it does provide a way to add documentation strings accessible at runtime, although in forms separate from the defining forms.

I still prefer documentation strings a la Common Lisp and Emacs Lisp and the example I gave. They're right inside the defining form, which means that the identifier doesn't have to be repeated, and they're less verbose.

It should be easy to add them to lambda, case-lambda, define-syntax, etc. without affecting existing code. Just off the top of my head:

(define section-element
  (let ((associate (association-procedure string=? car)))
    (lambda (section element-name)
      "Return element named `element-name' from `section'."
      (let ((association (associate element-name section)))
        (and association
             (cdr association))))))

(define convui-regexp
  (case-lambda
   "Match regular expression `pattern', obeying `case-fold?' (default #f), and
use optional `lift' to lift the result."
   ((pattern) (convui-regexp pattern #f))
   ((pattern case-fold?)
    (convui-regexp pattern case-fold? (lambda (r s) (re-match-extract s r 0))))
   ((pattern case-fold? lift)
    ...)))

(define-syntax assert
  (syntax-rules ()
   "Return the symbol `passed' when the test passes.  The unit test
framework..."
    ((_ (operator argument ...))
     ...)
    ((_ (operator argument ...) error-values ...)
     ...)
    ((_ expression)
     ...)
    ((_ expression error-values ...)
     ...)))

And for defining forms that haven't been modified to accept documentation strings, e.g. when making a top-level binding to a list, a comment form like this might work:

or even:

The goals I'd like to achieve:

  • documentation strings accessible at runtime, e.g. through the REPL or editor tools
  • terse definitions
  • existing code runs unchanged
Thanks for considering the idea.

Marc Nieper-Wißkirchen

unread,
Apr 5, 2020, 4:55:20 PM4/5/20
to scheme-re...@googlegroups.com, John Cowan
Am So., 5. Apr. 2020 um 21:44 Uhr schrieb Arthur A. Gleckler <a...@speechcode.com>:

On Sun, Apr 5, 2020 at 12:45 AM Arthur A. Gleckler <a...@speechcode.com> wrote:

Sorry, I don't yet know anything about Scribble. I'll try to read up on it.

Okay, I found 4.5 In-Source Documentation and the In-Source Documentation section of Submodules. The latter gives examples. It is geared to support Racket's contract system, but it does provide a way to add documentation strings accessible at runtime, although in forms separate from the defining forms.

One advantage of Racket's model is that it is extensible with Scheme code and that it is expanded and run at different phases than the library itself is expanded and run.

Maybe Alex Shinn can explain Chibi's version of Scribble, which is also used to generate package documentation for snow-fort.org.


For case-lambda, this would break code that does not understand documentation strings.
Likewise.

And for defining forms that haven't been modified to accept documentation strings, e.g. when making a top-level binding to a list, a comment form like this might work:

or even:


The latter form allows "comment" to be implemented as a macro; the former version doesn't. Thus, the latter form is preferable.

A new "define" syntax with three arguments could also be defined.

Marc

The goals I'd like to achieve:

  • documentation strings accessible at runtime, e.g. through the REPL or editor tools
  • terse definitions
  • existing code runs unchanged
Thanks for considering the idea.

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

Arthur A. Gleckler

unread,
Apr 5, 2020, 4:59:11 PM4/5/20
to scheme-re...@googlegroups.com, John Cowan
On Sun, Apr 5, 2020 at 1:55 PM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:

For case-lambda, this would break code that does not understand documentation strings. 

Of course, but existing code wouldn't break.  People who wanted to use this form of case-lambda would make sure to import it.
 
The latter form allows "comment" to be implemented as a macro; the former version doesn't. Thus, the latter form is preferable.

Yes, I prefer the latter form, anyway.
 
A new "define" syntax with three arguments could also be defined.

Yes, that's a good point. 

Alex Shinn

unread,
Apr 6, 2020, 4:01:58 AM4/6/20
to scheme-re...@googlegroups.com, John Cowan
On Mon, Apr 6, 2020 at 5:55 AM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
Am So., 5. Apr. 2020 um 21:44 Uhr schrieb Arthur A. Gleckler <a...@speechcode.com>:

On Sun, Apr 5, 2020 at 12:45 AM Arthur A. Gleckler <a...@speechcode.com> wrote:

Sorry, I don't yet know anything about Scribble. I'll try to read up on it.

Okay, I found 4.5 In-Source Documentation and the In-Source Documentation section of Submodules. The latter gives examples. It is geared to support Racket's contract system, but it does provide a way to add documentation strings accessible at runtime, although in forms separate from the defining forms.

One advantage of Racket's model is that it is extensible with Scheme code and that it is expanded and run at different phases than the library itself is expanded and run.

Maybe Alex Shinn can explain Chibi's version of Scribble, which is also used to generate package documentation for snow-fort.org.

The basic idea is that comment lines immediately starting with > (and no space) are used for documentation of the _following_ definition:

;;> Returns true iff str is equal to the empty string "".
(define (string-null? str) (equal? str ""))

;;> The ratio of a circle's circumference to its diameter.
(define pi 3.14159)

The docs can use scribble markup, and can optionally start with a signature (otherwise inferred from source):

;;> \procedure{(string-null? str)}.
;;> Returns true iff \var{str} is equal to the empty string \scheme{""}.
(define (string-null? str) (equal? str ""))

If you want to summarize a group of definitions, instead put the documentation after all definitions and start the comment with / on the final line.  It then applies to all _preceding_ definitions up to any previous comment:

(define black-escape (make-simple-escape-procedure 30))
(define red-escape (make-simple-escape-procedure 31))
...
(define white-escape (make-simple-escape-procedure 37))

;;> Return a string consisting of an ANSI escape code to select the
;;> specified text color.
;;/

You can use \section{name} for section grouping and \example{expr} to include an expression along with it's evaluated result in the docs.

The (chibi doc) module provides the (print-procedure-docs proc) to output the documentation for a procedure at runtime, and (print-module-docs module-name) to print the docs for a whole module, with options to display as HTML (used by the tooling on snow-fort.org).  This parses the source at runtime so there is zero overhead if not using (chibi doc).

--
Alex

Arne Babenhauserheide

unread,
Apr 6, 2020, 7:22:56 AM4/6/20
to scheme-re...@googlegroups.com, John Cowan

Alex Shinn <alex...@gmail.com> writes:
> The basic idea is that comment lines immediately starting with > (and no
> space) are used for documentation of the _following_ definition:
>
> ;;> Returns true iff str is equal to the empty string "".
> (define (string-null? str) (equal? str ""))
>
> ;;> The ratio of a circle's circumference to its diameter.
> (define pi 3.14159)

When you parse comments, you lose the property that code is data. If you
(read) your code and process it, the documentation is gone.

When your documentation is part of the code tree, you can process the
code and still have the documentation.

Therefore I strongly prefer a model which includes documentation in the
code instead of adding something on top that’s parsed by a separate
tool.

I’ve experienced both styles first hand — docstrings as part of the
definition in Python and Guile and documentation in form of comments in
Java and C, and having docstrings as first class elements in your code
is much cleaner and makes it easier for programmers to build on them.

(define (string-null? str)
"Returns true iff str is equal to the empty string"
(equal? str ""))

elf

unread,
Apr 6, 2020, 7:53:51 AM4/6/20
to scheme-re...@googlegroups.com
How, then, does one distinguish between a docstring and a string return value? This gets especially hairy if one allows multiline docstrings.

There is also size overhead for non-parsed comments, as they must necessarily be included in the code proper. This can lead to some debugging issues, especially in a case like that above.

Anything that I can think of to solve these involves making any reader system so much more complicated as to negate any benefit. Please tell me if I'm wrong here, of course.

(This is not a theoretical set of problems, either. When hacking on scwm 20 years ago, this came up where the guile docstring was being incorrectly added to the string return value, iirc. Something like that, unfortunately I do not remember exactly atm but I can dig through my archive for the concrete case if desired.)

I would strongly prefer either a separate read syntax (as in the ;;> above), a macro definition (docstring-lambda that will parse out the relevant bits), or a separate documentation lib of some sort to avoid these sorts of issues.

-elf

Marc Nieper-Wißkirchen

unread,
Apr 6, 2020, 8:13:51 AM4/6/20
to scheme-re...@googlegroups.com
Addressing Arne's and elf's concerns, I think we can learn a lot from Racket's model, which has documentation (in some form of Scheme code) and the documented code itself in the same file while not producing any runtime overhead.

In the R6RS library model as being advertised by Ghuloum and Dybvig (I would talk about the R7RS library model but it isn't yet as sophisticated as it doesn't have to be yet because the only macro transformer we have is syntax-rules), a library can be *expanded*, *visited* and *invoked*.  When a library appears in an import declaration (or in a clause of cond-expand), it will be located and expanded (if not already expanded).  The expansion will reveal all top-level definitions and whether they are variables or keywords.  The expansion will produce two procedures, a visit procedure and an invoke procedure.  When any keyword binding is needed from the library, the visit procedure will be called (if not already called) to produce the actual transformer of the keyword.  When any variable binding is needed from the same library, the invoke procedure will be called (if not already called) to produce the actual value of the variable.  This model, in particular, ensures that only code that is needed is to be evaluated and, say, macro transformers in a library do not add to the runtime of the final program.

In-file documentation can be added to this model by adding a procedure to any expanded version of a library that actually registers the documentation with the Scheme system.  A repl would call this procedure, a compiled program wouldn't.

Marc

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

elf

unread,
Apr 6, 2020, 9:07:28 AM4/6/20
to scheme-re...@googlegroups.com
If I understand this correctly, this only is relevant for code with docstrings as part of a library. Not all documented code is necessarily part of a library, I think.

This appears to me, on the surface, as a very complex way to get around fairly simple reader syntax or a lightweight macro or lambda properties solution... I do not see the need for so much extra gunk. (One of my objections to R6 at the time.)

-elf

Marc Nieper-Wißkirchen

unread,
Apr 6, 2020, 9:13:37 AM4/6/20
to scheme-re...@googlegroups.com
Am Mo., 6. Apr. 2020 um 15:07 Uhr schrieb elf <e...@ephemeral.net>:
If I understand this correctly, this only is relevant for code with docstrings as part of a library. Not all documented code is necessarily part of a library, I think.

I thought this as well, but then I began to wonder how much code you want to document that is not part of a library. Moreover, how do you access such documentation?
 
This appears to me, on the surface, as a very complex way to get around fairly simple reader syntax or a lightweight macro or lambda properties solution... I do not see the need for so much extra gunk. (One of my objections to R6 at the time.)

I didn't want to advertise to copy Racket's approach verbatim. But we should learn from the lessons they have learned. A large system wants so much extra "gunk", a small system wants to live without, so we have to find a solution that is easy to start with and whose model is sophisticated enough for a large system that separates phases, etc.

Note that a lambda properties solution isn't enough; non-procedure globals and defaults also want to be documented.

Marc


Arne Babenhauserheide

unread,
Apr 6, 2020, 9:47:20 AM4/6/20
to scheme-re...@googlegroups.com

elf <e...@ephemeral.net> writes:
> How, then, does one distinguish between a docstring and a string return value? This gets especially hairy if one allows multiline docstrings.

What Guile does: if the string is the only part of the body, then there
is no docstring: it is not a docstring but a return value.

> Anything that I can think of to solve these involves making any reader system so much more complicated as to negate any benefit. Please tell me if I'm wrong here, of course.

The above ensures that a string is only a docstring if using it as a
docstring has no side-effect.

Parsing a docstring is the same as parsing any other string.

It is already allowed in r7rs to have a string as first expression in
the body, so nothing changes in the syntax (different from the
comment-version where there’s new syntax that’s hidden in comments).

What changes is that if the first form in define is a string, and it is
not the last form, then this string becomes the docstring.

The long form is
(set-procedure-property! fun 'documentation "the docstring")

But since there is obviously no consensus about this (strong preferences
both ways), first creating SRFIs might be a better way forward.

Best wishes,
Arne

Marc Nieper-Wißkirchen

unread,
Apr 6, 2020, 9:58:03 AM4/6/20
to scheme-re...@googlegroups.com
Am Mo., 6. Apr. 2020 um 15:47 Uhr schrieb Arne Babenhauserheide <arne...@web.de>:

elf <e...@ephemeral.net> writes:
> How, then, does one distinguish between a docstring and a string return value? This gets especially hairy if one allows multiline docstrings.

What Guile does: if the string is the only part of the body, then there
is no docstring: it is not a docstring but a return value.

> Anything that I can think of to solve these involves making any reader system so much more complicated as to negate any benefit. Please tell me if I'm wrong here, of course.

The above ensures that a string is only a docstring if using it as a
docstring has no side-effect.

Parsing a docstring is the same as parsing any other string.

It is already allowed in r7rs to have a string as first expression in
the body, so nothing changes in the syntax (different from the
comment-version where there’s new syntax that’s hidden in comments).

A string as the first expression is not allowed if you want to have internal definitions in the body.

 

What changes is that if the first form in define is a string, and it is
not the last form, then this string becomes the docstring.

The long form is
(set-procedure-property! fun 'documentation "the docstring")

But since there is obviously no consensus about this (strong preferences
both ways), first creating SRFIs might be a better way forward.

Isn't it more effective to seek consensus first? Otherwise, we have several competing SRFIs and still have no consensus.

At least, we can try to agree on the technical advantages and disadvantages and make up our minds accordingly.

Marc

Best wishes,
Arne


--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

Arne Babenhauserheide

unread,
Apr 6, 2020, 10:16:27 AM4/6/20
to scheme-re...@googlegroups.com

Arne Babenhauserheide <arne...@web.de> writes:

> elf <e...@ephemeral.net> writes:
>> How, then, does one distinguish between a docstring and a string return value? This gets especially hairy if one allows multiline docstrings.
>
> What Guile does: if the string is the only part of the body, then there
> is no docstring: it is not a docstring but a return value.

Addition: Parsing comments is harder than this: Any comment parser must
include a full string-parser, because it must distinguish between
comments and multiline-strings. Example:

"foo \\\"
;;> foo "
(define (foo) #f)

>> Anything that I can think of to solve these involves making any reader system so much more complicated as to negate any benefit. Please tell me if I'm wrong here, of course.
>
> The above ensures that a string is only a docstring if using it as a
> docstring has no side-effect.

Addition: I do not know a clean way to document objects that are defined
in a way which does not allow empty expressions.

(define foo "doc" #f) ;; <- syntax error.
(define (foo) "doc" #f) ;; <- valid and nice.

Marc Nieper-Wißkirchen

unread,
Apr 6, 2020, 10:28:48 AM4/6/20
to scheme-re...@googlegroups.com
Am Mo., 6. Apr. 2020 um 16:16 Uhr schrieb Arne Babenhauserheide <arne...@web.de>:

Arne Babenhauserheide <arne...@web.de> writes:

> elf <e...@ephemeral.net> writes:
>> How, then, does one distinguish between a docstring and a string return value? This gets especially hairy if one allows multiline docstrings.
>
> What Guile does: if the string is the only part of the body, then there
> is no docstring: it is not a docstring but a return value.

Addition: Parsing comments is harder than this: Any comment parser must
include a full string-parser, because it must distinguish between
comments and multiline-strings. Example:

"foo \\\"
;;> foo "
(define (foo) #f)

The Scheme reader has to cope with comments and strings anyway, so the necessary code should already be there.

What is probably not much work is the following: Change Scheme's "read" into a, say "read*" procedure that reads Scheme datums but Scheme datums wrapped into two special records, namely, say, #<source-position <datum> <pos>> and #<comment <datum> <comment>>.  The #<source-position ...> wrap can appear around every datum (and subdatum) whenever the reader can communicate the source position of the datum; the #<comment ...> wrap will appear around every datum that is preceded by some special comment syntax.

This solves the problem of reading and having code and comments together. It doesn't answer the question of how to associate the comment with the expanded code. This is not independent of the expander as this snippet

;;> foo
(baz (foo) #f)

may expand into a definition.
 
>> Anything that I can think of to solve these involves making any reader system so much more complicated as to negate any benefit. Please tell me if I'm wrong here, of course.
>
> The above ensures that a string is only a docstring if using it as a
> docstring has no side-effect.

Addition: I do not know a clean way to document objects that are defined
in a way which does not allow empty expressions.

(define foo "doc" #f) ;; <- syntax error.
(define (foo) "doc" #f) ;; <- valid and nice.

If I understand Arthur correctly, he suggests extending the `define' form to take more than two arguments when not defining a procedure.
 

Best wishes,
Arne
--
Unpolitisch sein
heißt politisch sein
ohne es zu merken

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

Arne Babenhauserheide

unread,
Apr 6, 2020, 1:07:03 PM4/6/20
to scheme-re...@googlegroups.com

Marc Nieper-Wißkirchen <marc....@gmail.com> writes:

> Am Mo., 6. Apr. 2020 um 15:47 Uhr schrieb Arne Babenhauserheide <
> A string as the first expression is not allowed if you want to have
> internal definitions in the body.

Ah, you’re right. This is a limitation Guile just lifted in 3.0. It was
one of the most jarring things when I learned Scheme.

>> What changes is that if the first form in define is a string, and it is
>> not the last form, then this string becomes the docstring.
>>
>> The long form is
>> (set-procedure-property! fun 'documentation "the docstring")
>>
>> But since there is obviously no consensus about this (strong preferences
>> both ways), first creating SRFIs might be a better way forward.
>>
>
> Isn't it more effective to seek consensus first? Otherwise, we have several
> competing SRFIs and still have no consensus.

Can we reach consensus when there are such strong opinions?

If we can, you’re right.

> At least, we can try to agree on the technical advantages and disadvantages
> and make up our minds accordingly.

That sounds good.

elf

unread,
Apr 6, 2020, 1:51:11 PM4/6/20
to scheme-re...@googlegroups.com
FWIW, i don't have a strong opinion; if everyone agrees to something else, i'm good with that. That said, i would prefer a solution that involves something simpler (like an extra reader syntax symbol, that can be handled in implementation dependent ways, if desired, but won't cause problems for any non-implementing implementations) to something that overhauls everything about how the reader itself works.

Regarding the non-library documentation use cases, top-level applications are not necessarily libraries (are often not libraries) but could require internal documentation; likewise, prototyping and testing code may not be packaged as a library but some form of docstring may be useful. I think it is a mistake to tightly couple the idea of docstrings with the idea of libraries.

(i'm sorry i'm responding out of order but i'm on very limited access at the moment with extremely flaky internet and even flakier hardware, so i'm trying to condense things.)

Is there an inherent problem with a form something like:
(define (foo x)
(docstring "x get fooed")
...
)

with docstring as either a macro or a procedure? (macro is probably better, although i think, on reflection, that this will boil down to a macro redefinition of define in order to work properly without incurring per call overhead, which may not be ideal...)

-elf

Edwin Watkeys

unread,
Apr 6, 2020, 2:07:09 PM4/6/20
to scheme-re...@googlegroups.com
Hi all,

As a general comment, I’d like to mention that it may be worth thinking about object metadata more broadly and considering whether the larger issue is worth pondering, and if so, how documentation could be implemented within a more general metadata framework.

I menton this for two reasons: Clojure uses its metadata system, which has been put to many valuable uses, to implement doc strings IIRC, and I think that, if doc strings are to be supported, names bound to non-procedures (and/or those non-procedure objects themselves) should be supported.

Regards,
Edwin

> On Apr 6, 2020, at 13:51, elf <e...@ephemeral.net> wrote:
>
> FWIW, i don't have a strong opinion; if everyone agrees to something else, i'm good with that. That said, i would prefer a solution that involves something simpler (like an extra reader syntax symbol, that can be handled in implementation dependent ways, if desired, but won't cause problems for any non-implementing implementations) to something that overhauls everything about how the reader itself works.
> --
> You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/scheme-reports-wg2/755B2E26-8EC5-4E2C-BCAE-0CAF738F20E9%40ephemeral.net.

Per Bothner

unread,
Apr 6, 2020, 3:08:19 PM4/6/20
to scheme-re...@googlegroups.com
On 4/6/20 7:28 AM, Marc Nieper-Wißkirchen wrote:
> What is probably not much work is the following: Change Scheme's "read" into a, say "read*" procedure that reads Scheme datums but Scheme datums wrapped into two special records, namely, say, #<source-position <datum> <pos>> and #<comment <datum> <comment>>.  The #<source-position ...> wrap can appear around every datum (and subdatum) whenever the reader can communicate the source position of the datum; the #<comment ...> wrap will appear around every datum that is preceded by some special comment syntax.

FWIW: The Kawa reader will read each pair (cons) as a PairWithPosition, which is a subclass of Pair
augmented with position information. This makes it somewhat transparent to code that doesn't
care/know about positions.

How do you get the position of a string literal? Well, (almost) every string literal is the
car of some list so the position of the string is the position of the car.
Exceptions are dotted-lists (which are rarely used in Scheme code, except for lambda rest
parameters), and top-level forms (which Kawa deals with by wrapping them in a pseudo-begin form).

--
--Per Bothner
p...@bothner.com http://per.bothner.com/

Arne Babenhauserheide

unread,
Apr 6, 2020, 4:11:03 PM4/6/20
to scheme-re...@googlegroups.com

Edwin Watkeys <e...@poseur.com> writes:
> As a general comment, I’d like to mention that it may be worth thinking about object metadata more broadly and considering whether the larger issue is worth pondering, and if so, how documentation could be implemented within a more general metadata framework.
>
> I menton this for two reasons: Clojure uses its metadata system, which has been put to many valuable uses, to implement doc strings IIRC, and I think that, if doc strings are to be supported, names bound to non-procedures (and/or those non-procedure objects themselves) should be supported.

Regarding this, Guile implements docstrings as a special case of object
properties:


(define my-property (make-object-property))
(set! (PROPERTY OBJ) VAL)
;; A single object property created by ‘make-object-property’ can
;; associate distinct property values with all Scheme values that are
;; distinguishable by ‘eq?’ (ruling out numeric values).
;;
;; Internally, object properties are implemented using a weak key hash
;; table. This means that, as long as a Scheme value with property values
;; is protected from garbage collection, its property values are also
;; protected. When the Scheme value is collected, its entry in the
;; property table is removed and so the (ex-) property values are no longer
;; protected by the table.

(procedure-documentation proc) ; => docstring
(procedure-property proc 'documentation); => docstring
(procedure-properties proc) ; Return the properties associated with PROC, as an association list.
;; and
(set-procedure-properties! proc alist)
(set-procedure-property! proc key value)
;; for other objects
(object-properties obj) ; Return OBJ’s property list.
(object-property obj key)
(set-object-properties! obj alist)
(set-object-property! obj key value)


If you define with docstring
(define (foo) "bar" #t)
you just set the 'documentation property.
(procedure-properties foo)
$1 ((name . foo) (documentation . "bar"))

And you can set the properties directly:
(define (foo) #((foo . bar)) #t)
(procedure-properties foo)
$2 = ((name . foo) (foo . bar))

which can be combined with docstrings:

(define (foo) "myfoo" #((foo . bar)) #t)
(procedure-properties foo)
$3 = ((name . foo) (documentation . "myfoo") (foo . bar))

But in case of conflict, the properties win over the docstring:
(define (foo) "myfoo" #((foo . bar) (documentation . "yourfoo")) #t)
(procedure-properties foo)
$4 = ((name . foo) (foo . bar) (documentation . "yourfoo"))

Alaric Snell-Pym

unread,
Apr 7, 2020, 5:39:31 AM4/7/20
to scheme-re...@googlegroups.com
On 06/04/2020 19:07, Edwin Watkeys wrote:
> Hi all,
>
> As a general comment, I’d like to mention that it may be worth thinking about object metadata more broadly and considering whether the larger issue is worth pondering, and if so, how documentation could be implemented within a more general metadata framework.
>
> I menton this for two reasons: Clojure uses its metadata system, which has been put to many valuable uses, to implement doc strings IIRC, and I think that, if doc strings are to be supported, names bound to non-procedures (and/or those non-procedure objects themselves) should be supported.

I'm interested in this, too.

One of my back-burner thought experiments is a language design whereby
modules aren't just maps from names to things, but something more
general: lists of Prolog-style predicate-logic claims, which can include
"name X is bound to value Y" or "name X is bound to macro-expander Y" as
well as "value Y is a function with these argument/return types" or
"value Y has these pre/post conditions" or "value Y has intended usage
Z, a string of English text".

Note that I propose attaching all this metadata to the values, not the
names - because a name might be bound to a list, and then documentation
attached to the elements of the list, etc.

(I'm hoping I can use that to implement some kind of generic function
system, which is an example of a means of binding values to names that's
more complicated than just "name = value", but I've not figured out the
details of that yet!)

On another note: In the syntax that generates these things, I'm also
toying with having deeper metadata than just a bland "comment" form.
Examples would include:

- Intended usage (useful for exported bindings)

- Example usage (not just a string like "(+ 2 1) => 3", more like an
executable testcase that can be rendered in documentation as well)

- Implementation explanation (useful to wrap around ANY sub-expression,
not just top-level definitions) - let's give this a nice short name like
'?':

(define (total-power-used)
(+ power-used-by-motor
power-used-by-lights
(? "The power used by the light on the motor counts in both "

"the above totals, so we'd count it twice otherwise"
(- power-used-by-motor-light))))

- A FIXME, documenting work that can be done to improve the code, but
which is not required to release it:

(define (find-things-to-do-in-priority-order)
(FIXME "Use a heap to store the task list in order rather than
sorting it every time"
(sort *current-tasks*)))

- A marker for placeholders/stubs for not yet implemented things, which
needs to be done before the code can be released:

(define (check-access-is-permitted user action object)
(PLACEHOLDER "Interface into corporate LDAP server to check group
membership; for now, anything goes"
#t))

...a CI system can issue warnings for placeholders in branch code and
refuse to accept a merge to master with any placeholders in, for instance.

On another note, to avoid too much nesting caused by all these comment
blocks that wrap a value, my syntax also includes a syntactic sugar for
them - "(A . X): Y" -> (A . (X . (Y)))", so you can write this:

(define (some-func)
(? "foo"):
...body...)

to mean:

(define (some-func)
(? "foo"
...body...))

...without needing an extra set of parens around the body that cause it
to be indented further. A reader extension along those lines might be a
reasonable compromise between the (comment STRING BODY) and "STRING"
BODY camps?

> Regards,
> Edwin

Thanks,

--
Alaric Snell-Pym (M7KIT)
http://www.snell-pym.org.uk/alaric/

signature.asc

elf

unread,
Apr 7, 2020, 6:48:10 AM4/7/20
to scheme-re...@googlegroups.com
um, alaric, i think this runs into the halting paradox, or something very much like it, in the general case :)

-elf

Alaric Snell-Pym

unread,
Apr 7, 2020, 8:15:49 AM4/7/20
to scheme-re...@googlegroups.com
On 07/04/2020 11:47, elf wrote:
> um, alaric, i think this runs into the halting paradox, or something very much like it, in the general case :)

If you allow *rules* as well as *assertions* in your modules, then
extracting the semantics / metadata of a module might be
non-terminating, is that what you mean?

Well, don't do that then ;-)

>
> -elf
signature.asc

elf

unread,
Apr 7, 2020, 8:53:58 AM4/7/20
to scheme-re...@googlegroups.com
define the difference between deriving assertions and rules? in some of the cases you mentioned, this would be a problem, i think. i could, of course, be wrong. can any of the formal grammarians put in 2c?

-elf

Alaric Snell-Pym

unread,
Apr 7, 2020, 10:55:34 AM4/7/20
to scheme-re...@googlegroups.com
On 07/04/2020 13:53, elf wrote:
> define the difference between deriving assertions and rules? in some of the cases you mentioned, this would be a problem, i think. i could, of course, be wrong. can any of the formal grammarians put in 2c?

Sure! If you have an input syntax like, for example:

(define-library (example addition)
(import (scheme base))
(export add)
(begin
(define (add x y) "Return the sum of x and y" (+ x y))
(expect (add 1 2) 3)))

...then the "first-class module value" that produces when separately
compiled or whatever might, assuming a fancy type-infering compiler,
consist of these assertions:

(binding 'add <closure value X>)
(type <closure value X> (-> (number number) (number)))
(argument-names <closure value X> '(x y))
(documentation <closure value X> "Return the sum of x and y")
(example <closure value X> (1 2) (3))

These are just "statements of fact" rendered as tuples. All of them can
be derived from the module definition above using existing techniques,
and extracting them from the assertion list is easy - we can ask "What
names are bound in this module?" by finding matches for `(binding
<something> <something>)` in O(N) time, etc.

As opposed to a more conventional implementation of a module which might
just return a kind of alist of bindings, from names to either values or
macro transformers. Instead, we return a list of these assertions about
things.

However, languages like Prolog that operate in this kind of model extend
those simple lists of assertions with the ability to also specify
"rules", such as:

(rule (parent X Y) (son Y X))
(rule (parent X Y) (daughter Y X))
(rule (ancestor X Y) (parent X Y))
(rule (ancestor X Y)
(parent X Z)
(ancestor Z Y))

...which, when combined with some facts like:

(son adam steve)
(son bob steve)
(daughter carol adam)
(son bob carol)

...then lets the system answer questions such as:

(ancestor bob <something>)

...with a list of possible somethings: carol, adam, and steve (but not
bob), by repeatedly applying the rules and assertions until it can find
all the answers. However, it's possible to encode arbitrary computations
in that, and so answering questions in the presence of arbitrary rules
such as the above might, indeed, not terminate.

Although I *am* interested in the possibility of allowing rules as part
of a type system in this manner, because it does let you do some fun
stuff as long as you can find ways to deal with compilation possibly not
terminating, that's outside the scope of using assertions alone as an
example of a more general metadata system that might be used in the
context of doc strings, the original intent of this thread :-)

>
> -elf
>

TTFN,
signature.asc

elf

unread,
Apr 7, 2020, 11:19:07 AM4/7/20
to scheme-re...@googlegroups.com
so, related to previous discussion in a diff thread here... what happens when a procedure can return #f, eol, or a string? even the statements of fact can be arbitrarily complex and (possibly) misleading without further assertions for explanation, which themselves require further assertions, etc.

this becomes even hairier if we have, say, map, or even better, fold... i do not believe any generic iterators or generic generators (if im using the terminology correctly) could be reasonably asserted in such a fashion....

how would macros be handled? macro introspection can even get weirder...

it's a fun idea, i think, but runs into too many problems in a system like scheme. again, just my 2c. in a strongly typed scheme variant it would be interesting to see.

-elf

elf

unread,
Apr 7, 2020, 11:19:14 AM4/7/20
to scheme-re...@googlegroups.com
so, related to previous discussion in a diff thread here... what happens when a procedure can return #f, eol, or a string? even the statements of fact can be arbitrarily complex and (possibly) misleading without further assertions for explanation, which themselves require further assertions, etc.

this becomes even hairier if we have, say, map, or even better, fold... i do not believe any generic iterators or generic generators (if im using the terminology correctly) could be reasonably asserted in such a fashion....

how would macros be handled? macro introspection can even get weirder...

it's a fun idea, i think, but runs into too many problems in a system like scheme. again, just my 2c. in a strongly typed scheme variant it would be interesting to see.

-elf

Alaric Snell-Pym

unread,
Apr 7, 2020, 1:51:07 PM4/7/20
to scheme-re...@googlegroups.com
On 07/04/2020 13:53, elf wrote:
> define the difference between deriving assertions and rules? in some of the cases you mentioned, this would be a problem, i think. i could, of course, be wrong. can any of the formal grammarians put in 2c?

Marc Nieper-Wißkirchen

unread,
Apr 7, 2020, 2:01:37 PM4/7/20
to scheme-re...@googlegroups.com
Am Mo., 6. Apr. 2020 um 19:07 Uhr schrieb Arne Babenhauserheide <arne...@web.de>:

Marc Nieper-Wißkirchen <marc....@gmail.com> writes:

> Am Mo., 6. Apr. 2020 um 15:47 Uhr schrieb Arne Babenhauserheide <
> A string as the first expression is not allowed if you want to have
> internal definitions in the body.

Ah, you’re right. This is a limitation Guile just lifted in 3.0. It was
one of the most jarring things when I learned Scheme.

Neither in the R6RS, nor in the R7RS this restriction was lifted.

I think every modern Scheme could cope with an extension that allows expressions before definitions (as they are allowed in programs).  A procedure body like `(<expr> <definition><expr>)' would be equivalent to `((define-values <dummy-var> <expr>) <definition> <expr>)'.  (Cf the semantics of a program in the R6RS.)


>> What changes is that if the first form in define is a string, and it is
>> not the last form, then this string becomes the docstring.
>>
>> The long form is
>> (set-procedure-property! fun 'documentation "the docstring")
>>
>> But since there is obviously no consensus about this (strong preferences
>> both ways), first creating SRFIs might be a better way forward.
>>
>
> Isn't it more effective to seek consensus first? Otherwise, we have several
> competing SRFIs and still have no consensus.

Can we reach consensus when there are such strong opinions?

If we can, you’re right.

> At least, we can try to agree on the technical advantages and disadvantages
> and make up our minds accordingly.

That sounds good.

Before we think of any syntax, we should answer the following questions:

(1) Is it enough to document top-level definitions or do we want to be able to attach documentation to any value?
(2) Should we make sure that there is absolutely no runtime overhead when documentation is added?
(3) Do we just want to document procedures or any value?
(4) Do we want to be able to document syntactic keywords?
(5) How is the documentation related to the library system?
(6) Do we want structured documentation or just plain strings?
(7) Shall the documentation follow the code-is-data principle of Scheme?
(8) ...

Please note that, say, a yes to question (6) or (7) does not preclude simple documentation strings as long as they can be mapped homomorphically to a more sophisticated system.

-- Marc

 

Best wishes,
Arne
--
Unpolitisch sein
heißt politisch sein
ohne es zu merken

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

Arne Babenhauserheide

unread,
Apr 7, 2020, 4:26:22 PM4/7/20
to scheme-re...@googlegroups.com

Marc Nieper-Wißkirchen <marc....@gmail.com> writes:
> Before we think of any syntax, we should answer the following questions:
>
> (1) Is it enough to document top-level definitions or do we want to be able
> to attach documentation to any value?
> (2) Should we make sure that there is absolutely no runtime overhead when
> documentation is added?
> (3) Do we just want to document procedures or any value?
> (4) Do we want to be able to document syntactic keywords?
> (5) How is the documentation related to the library system?
> (6) Do we want structured documentation or just plain strings?
> (7) Shall the documentation follow the code-is-data principle of Scheme?
(8) Do we want to be able to access the documentation at runtime?
(9) ?

Arne Babenhauserheide

unread,
Apr 7, 2020, 5:03:05 PM4/7/20
to scheme-re...@googlegroups.com

Arne Babenhauserheide <arne...@web.de> writes:

> Marc Nieper-Wißkirchen <marc....@gmail.com> writes:
>> Before we think of any syntax, we should answer the following questions:

My answers — mind that I’m mostly a user of Scheme, not an implementor.

>> (1) Is it enough to document top-level definitions or do we want to be able
>> to attach documentation to any value?

I think it should be possible to attach documentation to any *variable*.
This can include nested definitions.

Attaching documentation to anything might have too high costs elsewhere.

>> (2) Should we make sure that there is absolutely no runtime overhead when
>> documentation is added?

I think "absolutely no" is too strong a statement. Even the increase in
filesize of the sources can add minimal runtime overhead.

I think it’s important that the system also works, if I only have
bytecode around and not the actual sources (because I don’t want to
force the system to switch to slower source-parsing when I access
docstrings: That would disincentivize using that information).

However keep in mind that we’re talking about r7rs large here. There
could easily be specialized schemes with harsher resource limitations
that don’t implement this.

>> (3) Do we just want to document procedures or any value?

I think any value, but I’m currently happy with the Guile approach of
providing a setter for properties.

I’m pretty sure, however, that I would not want to go back after having
used a define form for variables that allows me to attach documentation
to variables that don’t point to procedures :-)

(define wants-strange-behavior
"irregular and wonderful if set to #t"
#f)

>> (4) Do we want to be able to document syntactic keywords?

Do you mean the results of define-syntax?

>> (5) How is the documentation related to the library system?

I’m not sure. What do you think?

>> (6) Do we want structured documentation or just plain strings?

Both: structured documentation proved to be awesome for doctests, and
I’m sure more uses will be found if we add it.

>> (7) Shall the documentation follow the code-is-data principle of Scheme?

Yes. After having experienced that I can safely say that that’s awesome.

I started my doctests with python-like string-parsing, but the
property-based code-is-data doctests I have now (after people suggested
the properties) are simply much nicer.

> (8) Do we want to be able to access the documentation at runtime?

Yes, definitely. This is great, for example, to build self-testing
procedures, and for docstrings used in --help argument parsing.

(define (check-help args)
"--help shows this message and exits"
(when (member "--help" args)
(string-append "usage: "
(first args) " "
(procedure-documentation parse-args))
(exit 0)))

John Cowan

unread,
Apr 7, 2020, 5:55:50 PM4/7/20
to scheme-re...@googlegroups.com
Here's a little taxonomy that might help:

Should we be able to document:

Objects?
Variables?  Global only?
Variables bound to procedures? Global only?
Macros? Global only?
Record types?  Global only?


--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

Arne Babenhauserheide

unread,
Apr 7, 2020, 6:22:29 PM4/7/20
to scheme-re...@googlegroups.com

Alaric Snell-Pym <ala...@snell-pym.org.uk> writes:
> On another note: In the syntax that generates these things, I'm also
> toying with having deeper metadata than just a bland "comment" form.
> Examples would include:
>
> - Intended usage (useful for exported bindings)
>
> - Example usage (not just a string like "(+ 2 1) => 3", more like an
> executable testcase that can be rendered in documentation as well)
>
> - Implementation explanation (useful to wrap around ANY sub-expression,
> not just top-level definitions) - let's give this a nice short name like
> '?':

You might be interested in the doctest stuff I did:

(define (A)
"returns 'A"
#((tests (test-eqv 'A (A))
(test-assert #t)))
'A)

This already works in Guile and is quite similar to your ? idea, though
not as general.

'?' as name sounds dangerous to me, though: This is already used in
other places (it’s far too common). Basically every special character
for not-absolutely-core-operations like λ and '`, is dangerous, because
it increases complexity for human readers and newcomers.

The path to Haskell-type syntax complexity is short and we should think
thrice before adding anything.

This is why I like the approach in Guile to avoid increasing the number
of chars with special handling by having long-name functions and
re-using existing literal forms.

But that can only be used in define-level granularity right now, so your
form can do more.

> (define (total-power-used)
> (+ power-used-by-motor
> power-used-by-lights
> (? "The power used by the light on the motor counts in both "
>
> "the above totals, so we'd count it twice otherwise"
> (- power-used-by-motor-light))))

(why the multi-strings above, given that there are multi-line strings in
Scheme?)

> - A FIXME, documenting work that can be done to improve the code, but
> which is not required to release it:
>
> (define (find-things-to-do-in-priority-order)
> (FIXME "Use a heap to store the task list in order rather than
> sorting it every time"
> (sort *current-tasks*)))

FIXME looks like something I might want to use as a procedure- or
variable-name; why not fold it into the general metadata form?.

John Cowan

unread,
Apr 7, 2020, 6:23:42 PM4/7/20
to scheme-re...@googlegroups.com
Question mark is not a name, it's a question mark!   In other words:  Should we be able to document any object?  Should we be able to document any variable? etc.


--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

Arthur A. Gleckler

unread,
Apr 7, 2020, 6:55:25 PM4/7/20
to scheme-re...@googlegroups.com

On Tue, Apr 7, 2020 at 11:01 AM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:

Before we think of any syntax, we should answer the following questions:

(1) Is it enough to document top-level definitions or do we want to be able to attach documentation to any value?

Ideally, I'd like to be able to document all definitions, regardless of level.

I'd also like to be able to document values like procedures and syntax transformers independent of whether they have bindings or where they are bound. It's easy to add an optional slot to both that can hold a documentation string or a key that can be used to fetch one at runtime. For other types, keeping a hash table from values to documentation strings would not be expensive.

(2) Should we make sure that there is absolutely no runtime overhead when documentation is added?

"Absolutely no overhead" is extreme. There are ways to minimize the overhead.

(3) Do we just want to document procedures or any value?

All bindings as well as the values of procedures and syntax transformers.

(4) Do we want to be able to document syntactic keywords?

Yes.

(5) How is the documentation related to the library system?

Unrelated.

(6) Do we want structured documentation or just plain strings?

I'd rather not impose a specific syntax on everyone, but it would be nice to allow implementations to support it. Perhaps reader syntax like the following might work, for example to indicate the use of Markdown markup:

I'm not advocating this specific reader syntax, but just the idea that a string could be labeled concisely to identify what syntax was being used. Alternatively:

In this example, the language would have to treat documentation specially in defining forms in addition to or instead of bare strings.

(7) Shall the documentation follow the code-is-data principle of Scheme?

Absolutely. I would like to avoid magic comments, for example.

(8) …

Arthur A. Gleckler

unread,
Apr 7, 2020, 6:55:56 PM4/7/20
to scheme-re...@googlegroups.com

On Tue, Apr 7, 2020 at 2:55 PM John Cowan <co...@ccil.org> wrote:

Should we be able to document:

Objects?

Yes.

Variables? Global only?

Yes. Once the mechanism is in place, there doesn't seem to be a reason to restrict it to top-level definitions.

Variables bound to procedures? Global only?

Yes. No.

Macros? Global only?

Yes. No.

Record types? Global only?

Good idea, and yes. No.

John Cowan

unread,
Apr 7, 2020, 6:58:42 PM4/7/20
to scheme-re...@googlegroups.com
On Tue, Apr 7, 2020 at 6:55 PM Arthur A. Gleckler <a...@speechcode.com> wrote:

Variables? Global only?

Yes. Once the mechanism is in place, there doesn't seem to be a reason to restrict it to top-level definitions.


Ah, that leads to another question:  Let-bindings, or just definitions?



John Cowan          http://vrici.lojban.org/~cowan        co...@ccil.org
I could dance with you till the cows come home.  On second thought,
I'd rather dance with the cows when you come home.
        --Rufus T. Firefly

Arthur A. Gleckler

unread,
Apr 7, 2020, 7:06:09 PM4/7/20
to scheme-re...@googlegroups.com
On Tue, Apr 7, 2020 at 3:58 PM John Cowan <co...@ccil.org> wrote:
 
Ah, that leads to another question:  Let-bindings, or just definitions? 

I don't see a way to make adding doc strings to let bindings unobtrusive — except bindings of procedures, where there's a natural place to put the documentation string.

John Cowan

unread,
Apr 7, 2020, 7:55:18 PM4/7/20
to scheme-re...@googlegroups.com
That would be a documentation of the object rather than the variable; those ideas are disjoint, and you could have both.

 

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

Arthur A. Gleckler

unread,
Apr 7, 2020, 8:42:42 PM4/7/20
to scheme-re...@googlegroups.com
On Tue, Apr 7, 2020 at 4:55 PM John Cowan <co...@ccil.org> wrote:

On Tue, Apr 7, 2020 at 7:06 PM Arthur A. Gleckler <a...@speechcode.com> wrote:
 
I don't see a way to make adding doc strings to let bindings unobtrusive — except bindings of procedures, where there's a natural place to put the documentation string.

That would be a documentation of the object rather than the variable; those ideas are disjoint, and you could have both.
 
Yes, of course, but I don't see how to add documentation to a let binding itself and not make it obtrusive.

elf

unread,
Apr 8, 2020, 2:03:09 AM4/8/20
to scheme-re...@googlegroups.com
Am Mo., 6. Apr. 2020 um 19:07 Uhr schrieb Arne Babenhauserheide <
arne...@web.de>:

>
> Marc Nieper-Wißkirchen <marc....@gmail.com> writes:
>
> > Am Mo., 6. Apr. 2020 um 15:47 Uhr schrieb Arne Babenhauserheide <
> > A string as the first expression is not allowed if you want to have
> > internal definitions in the body.
>
> Ah, you’re right. This is a limitation Guile just lifted in 3.0. It was
> one of the most jarring things when I learned Scheme.
>

Neither in the R6RS, nor in the R7RS this restriction was lifted.

I think every modern Scheme could cope with an extension that allows
expressions before definitions (as they are allowed in programs). A
procedure body like `(<expr> <definition><expr>)' would be equivalent to
`((define-values <dummy-var> <expr>) <definition> <expr>)'. (Cf the
semantics of a program in the R6RS.)


> >> What changes is that if the first form in define is a string, and it is
> >> not the last form, then this string becomes the docstring.
> >>
> >> The long form is
> >> (set-procedure-property! fun 'documentation "the docstring")
> >>
> >> But since there is obviously no consensus about this (strong preferences
> >> both ways), first creating SRFIs might be a better way forward.
> >>
> >
> > Isn't it more effective to seek consensus first? Otherwise, we have
> several
> > competing SRFIs and still have no consensus.
>
> Can we reach consensus when there are such strong opinions?
>
> If we can, you’re right.
>
> > At least, we can try to agree on the technical advantages and
> disadvantages
> > and make up our minds accordingly.
>
> That sounds good.
>

Before we think of any syntax, we should answer the following questions:

(1) Is it enough to document top-level definitions or do we want to be able
to attach documentation to any value?

Documenting internal definitions seems to me to break one of the fundamental ideas of internal definitions. Documenting procedure parameters is part of documenting the procedure itself; documenting how everything works internally, externally exposed, removes the line between documentation and commenting.

(2) Should we make sure that there is absolutely no runtime overhead when
documentation is added?

I would say that documentation that incurs runtime overhead is problematic. Try debugging or optimising a tight loop with other people's documentation in small memory.

(3) Do we just want to document procedures or any value?

Exposed values are generally either immutable (constants or special symbols/types) or through what chicken (and others) calls parameters, ie procedures. In the first case, internal docstring-type documentation seems generally unnecessary; in the second, it's already covered.

(4) Do we want to be able to document syntactic keywords?

If they're externally exposed, yes.

(5) How is the documentation related to the library system?

It shouldn't be, unless either a) duplication of the same documentation system is desirable for top-level applications and libraries or b) all code should be in the form of a library. Both of these seem not so good to me.

(6) Do we want structured documentation or just plain strings?

Why does this matter? That is to say, a simple implementation should be able to treat the documentation as a simple string and still be understandable. Any structure should be something that can evolve as a set of keywords or the like, decided through srfis. Trying to define the total future behaviour seems to me to be a) limiting b) a design flaw c) something that will bite us later d) very unschemelike.

(7) Shall the documentation follow the code-is-data principle of Scheme?

Clarify this point, please? Documentation is a string. That a string can be evalable data is inherent in the concept of a scheme string, last I checked.

(8) ...

Please note that, say, a yes to question (6) or (7) does not preclude
simple documentation strings as long as they can be mapped homomorphically
to a more sophisticated system.

-- Marc




If i may add my (probably unwanted at this point) 2c...
there seems to me to be a worrisome trend towards overdefinition in many of the proposals and srfis as of late. This may be a misperception on my part; I have not been as active as i used to be. That said, my understanding of the concept of scheme (and one of the things that makes it so fun to work with and play with and implement and .... etc) is that there's an inherent simplicity, and much of the expressiveness and power stems from the freedom that simplicity brings. A simple system, requiring as few changes as possible, that will serve the purpose of internal run-time documentation, seems to me to be far superior to any over-engineered behemoth that requires changes to every part of the repl and compiler. Making something _extremely_ simple and then seeing how (or if) it evolves in community-useful ways may be a better path than trying to find all the ways that all of us could ever theoretically want to use it ... and it will be less limiting to the next generation who may come up with things that we never thought of.

I am not saying that a complex system is necessary bad or a simple one is necessarily good, just that it might be a good idea to follow a route that requires the least amount of future correction if it turns out not to be what we had hoped.

(I will be unavailable for the next few days due to Passover, so I'm sorry in advance for putting this point out at this time. I will try to catch up on the discussion on Saturday night my time. Only mentioning so you're all free to flame or disagree and not be worried that I got offended and left in a huff. :) )

-elf

>
> Best wishes,
> Arne
> --
> Unpolitisch sein
> heißt politisch sein
> ohne es zu merken
>
> --
> You received this message because you are subscribed to the Google Groups
> "scheme-reports-wg2" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to scheme-reports-...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/scheme-reports-wg2/87ftdgr1ai.fsf%40web.de
> .
>

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scheme-reports-wg2/CAEYrNrRZ0%3D8teAG%2BN3Xz7Hm3qyzNGzmsK8imWhGiS7ruemapkQ%40mail.gmail.com.

Arne Babenhauserheide

unread,
Apr 8, 2020, 2:55:00 AM4/8/20
to scheme-re...@googlegroups.com

John Cowan <co...@ccil.org> writes:

> Question mark is not a name, it's a question mark!

It can be used as the name of a variable or procedure:

(define (? arg) arg)
(? '?-is-the-name)

That is what I mean by name.

And there are many domains in which ? is a natural choice for a name. If
we have a standard procedure take it, all those domains are prevented
from using it.

imaginary example that first came to my head: (solve (> 5 ?))

Arne Babenhauserheide

unread,
Apr 8, 2020, 3:33:04 AM4/8/20
to scheme-re...@googlegroups.com

elf <e...@ephemeral.net> writes:
> Before we think of any syntax, we should answer the following questions:
>
> (1) Is it enough to document top-level definitions or do we want to be able
> to attach documentation to any value?
>
> Documenting internal definitions seems to me to break one of the fundamental ideas of internal definitions. Documenting procedure parameters is part of documenting the procedure itself; documenting how everything works internally, externally exposed, removes the line between documentation and commenting.

Whether the documentation of the inner definition is accessible outside
the outer definition is a second question.

You’re right there: I would not want this to be visible from outside the
outer definition.

(define (foo)
(define (bar) "keep the bar for visibility"
(display (procedure-documentation bar)) ;; should work
#t)
(bar))

(display (procedure-documentation bar)) ;; should not

> (2) Should we make sure that there is absolutely no runtime overhead when
> documentation is added?
>
> I would say that documentation that incurs runtime overhead is problematic. Try debugging or optimising a tight loop with other people's documentation in small memory.

To me it depends on the amount of runtime overhead. It does not have to
be overhead when calling the procedure. It could just be overhead when
loading the program (due to additional operations when the procedure is
defined).

Something that makes procedure-calls slower would be a no-go for me.

> (3) Do we just want to document procedures or any value?
>
> Exposed values are generally either immutable (constants or special symbols/types) or through what chicken (and others) calls parameters, ie procedures. In the first case, internal docstring-type documentation seems generally unnecessary; in the second, it's already covered.

There are mutable values. For example:

(define foo (list 'foo))
(list-set! foo 0 'bar)

> (7) Shall the documentation follow the code-is-data principle of Scheme?
>
> Clarify this point, please? Documentation is a string. That a string can be evalable data is inherent in the concept of a scheme string, last I checked.

A string needs to be parsed for evaluation. code-is-data would mean that
you for example have an alist.

> A simple system, requiring as few changes as possible, that will serve
> the purpose of internal run-time documentation, seems to me to be far
> superior to any over-engineered behemoth that requires changes to
> every part of the repl and compiler.

I think so, too.

Simplicity is the reason why I prefer Scheme over Python.

> I am not saying that a complex system is necessary bad or a simple one is necessarily good, just that it might be a good idea to follow a route that requires the least amount of future correction if it turns out not to be what we had hoped.
>
> (I will be unavailable for the next few days due to Passover, so I'm sorry in advance for putting this point out at this time. I will try to catch up on the discussion on Saturday night my time. Only mentioning so you're all free to flame or disagree and not be worried that I got offended and left in a huff. :) )

What’s passover?

Best wishes,
Arne

Alaric Snell-Pym

unread,
Apr 8, 2020, 4:50:13 AM4/8/20
to scheme-re...@googlegroups.com
On 07/04/2020 16:18, elf wrote:
> so, related to previous discussion in a diff thread here... what happens when a procedure can return #f, eol, or a string? even the statements of fact can be arbitrarily complex and (possibly) misleading without further assertions for explanation, which themselves require further assertions, etc.
>
> this becomes even hairier if we have, say, map, or even better, fold... i do not believe any generic iterators or generic generators (if im using the terminology correctly) could be reasonably asserted in such a fashion....

Those are all issues for the type system / inference algorithms of the
language... all I'm discussing here is a way of representing metadata
such as those type declarations, wherever they come from and whatever
form they may take!

Per Bothner

unread,
Apr 8, 2020, 6:18:15 AM4/8/20
to scheme-re...@googlegroups.com
On 4/7/20 11:54 PM, Arne Babenhauserheide wrote:
> And there are many domains in which ? is a natural choice for a name. If
> we have a standard procedure take it, all those domains are prevented
> from using it.

? is used for the Kawa conditional pattern-matching operator:

https://www.gnu.org/software/kawa/Conditionals.html#idm45230723555216

Example:

(if (? x::integer expression)
(+ x 1) ;; if pattern matches, x is bound
'invalid) ;; if pattern does not match, x is unbound
--
--Per Bothner
p...@bothner.com http://per.bothner.com/

Marc Nieper-Wißkirchen

unread,
Apr 8, 2020, 8:20:43 AM4/8/20
to scheme-re...@googlegroups.com
Am Mi., 8. Apr. 2020 um 01:55 Uhr schrieb John Cowan <co...@ccil.org>:


On Tue, Apr 7, 2020 at 7:06 PM Arthur A. Gleckler <a...@speechcode.com> wrote:
On Tue, Apr 7, 2020 at 3:58 PM John Cowan <co...@ccil.org> wrote:
 
Ah, that leads to another question:  Let-bindings, or just definitions? 

I don't see a way to make adding doc strings to let bindings unobtrusive — except bindings of procedures, where there's a natural place to put the documentation string.

That would be a documentation of the object rather than the variable; those ideas are disjoint, and you could have both.

I think this is one of the main points and one that hasn't mentioned explicitly a lot:

Do we want to document bindings or do we want to document values?

As far as simplicity goes, a single documentation system should either attach documentation to the former or to the latter.  In this regard, both options are orthogonal.

Attaching documentation to values seems to be the simpler approach at first, but it has a number of disadvantages:

(1) Documentation cannot be easily attached to non-structured values.  It has been suggested to use a weak hash-table, whose equality predicate would most likely be `eq?' or `eqv?'.  But what would happen if we documented SRFI 143 with its `fx-width' value, which may in some implementation be 24.  We do not want attach documentation to the number 24.

(2) In Scheme, a procedure object is in fact a closure.  Usually, we don't want to attach the documentation to the closure but to the underlying `lambda', which doesn't appear as a value, though.

(3) While documentation attached to values is easy to use at the REPL, code has to be run to get access to the values. In general, I am thinking of an intelligent IDE, we don't want code to be run before we retrieve documentation (and other meta-information).

Thus, I think what we really want is to attach documentation to bindings, that is to variables and keywords.

For define/define-syntax forms, we have already seen a few ways how extend their syntax so that documentation can be attached.  We can think of syntax how to attach documentation to formal arguments as well, which will not only cover the case of documenting procedure arguments but also of let(rec)(*)(-syntax)-bindings.

Marc



 

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scheme-reports-wg2/CALnw4LKC2mHCo8P93wC%2BOCsY%2B%2B7qMTJ5bDhJ%3Dr8AuAe0xTCe9g%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

Arthur A. Gleckler

unread,
Apr 8, 2020, 1:53:44 PM4/8/20
to scheme-re...@googlegroups.com
On Wed, Apr 8, 2020 at 5:20 AM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
 
(1) Documentation cannot be easily attached to non-structured values.  It has been suggested to use a weak hash-table, whose equality predicate would most likely be `eq?' or `eqv?'.  But what would happen if we documented SRFI 143 with its `fx-width' value, which may in some implementation be 24.  We do not want attach documentation to the number 24.

That's why you want to be able to attach documentation to variables as well as values.  While the implementation of documentation for bindings and for values is orthogonal, they are conceptually strongly related, which is why it's good to discuss them both at the same time.
 
(2) In Scheme, a procedure object is in fact a closure.  Usually, we don't want to attach the documentation to the closure but to the underlying `lambda', which doesn't appear as a value, though.

I disagree.  I want the documentation to be attached to the closure itself so that my IDE and REPL can ask for the documentation no matter what variable or data structure it is found in.
 
(3) While documentation attached to values is easy to use at the REPL, code has to be run to get access to the values. In general, I am thinking of an intelligent IDE, we don't want code to be run before we retrieve documentation (and other meta-information).

Actually, I've always thought that the runtime heap is the ultimate data structure for keeping meta-information, and that all the approximations that modern editors do with regular expressions, elaborate partial parsing systems, etc. are inferior to read — or to a version of read that records line numbers, source file pathnames, etc.  One of the wonderful things about Scheme is that I can load code even when it is unfinished, e.g. even when some variables are unbound and the code doesn't type check.  I can do experiments, run parts of the code, find out what identifiers there are, etc.  I would like to add the ability to extract documentation strings to that list of capabilities.

Marc Nieper-Wißkirchen

unread,
Apr 9, 2020, 2:40:18 AM4/9/20
to scheme-re...@googlegroups.com
Am Mi., 8. Apr. 2020 um 19:53 Uhr schrieb Arthur A. Gleckler <a...@speechcode.com>:
On Wed, Apr 8, 2020 at 5:20 AM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
 
(1) Documentation cannot be easily attached to non-structured values.  It has been suggested to use a weak hash-table, whose equality predicate would most likely be `eq?' or `eqv?'.  But what would happen if we documented SRFI 143 with its `fx-width' value, which may in some implementation be 24.  We do not want attach documentation to the number 24.

That's why you want to be able to attach documentation to variables as well as values.  While the implementation of documentation for bindings and for values is orthogonal, they are conceptually strongly related, which is why it's good to discuss them both at the same time.

While both are about documentation, I am not sure how much the concepts are related. In any case, it would be good if whatever solution we find for either concept it does not step on the toes of the other.
 
 
(2) In Scheme, a procedure object is in fact a closure.  Usually, we don't want to attach the documentation to the closure but to the underlying `lambda', which doesn't appear as a value, though.

I disagree.  I want the documentation to be attached to the closure itself so that my IDE and REPL can ask for the documentation no matter what variable or data structure it is found in.

Through the closure value, the Scheme system could access the lambda form and thus any documentation attached to the code. Or do you really want to have different documentation on different closures of the same procedure?
 
 
(3) While documentation attached to values is easy to use at the REPL, code has to be run to get access to the values. In general, I am thinking of an intelligent IDE, we don't want code to be run before we retrieve documentation (and other meta-information).

Actually, I've always thought that the runtime heap is the ultimate data structure for keeping meta-information, and that all the approximations that modern editors do with regular expressions, elaborate partial parsing systems, etc. are inferior to read — or to a version of read that records line numbers, source file pathnames, etc.  One of the wonderful things about Scheme is that I can load code even when it is unfinished, e.g. even when some variables are unbound and the code doesn't type check.  I can do experiments, run parts of the code, find out what identifiers there are, etc.  I would like to add the ability to extract documentation strings to that list of capabilities.

The Racket editor's understanding of the source is very elaborate, I think.  It doesn't have to run the code, though. While at the REPL, this may be true, I don't think that you always want to run code to retrieve documentation.  Running general code has usually side effects.  Moreover, one run may differ from the next one.  We should also note that there are Scheme systems that are not REPL-centered but use the usual compile-and-run cycle.

Anyway, having the ability to attach information to values in running code (through weak hash tables, for example) is good to have, but does not render unnecessary documentation attached to bindings.  As said above, these are really orthogonal concepts.

Marc

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

Marc Nieper-Wißkirchen

unread,
Apr 9, 2020, 2:52:59 AM4/9/20
to scheme-re...@googlegroups.com


Am Mi., 8. Apr. 2020 um 08:03 Uhr schrieb elf <e...@ephemeral.net>:

[...]

If i may add my (probably unwanted at this point) 2c...
there seems to me to be a worrisome trend towards overdefinition in many of the proposals and srfis as of late. This may be a misperception on my part; I have not been as active as i used to be. That said, my understanding of the concept of scheme (and one of the things that makes it so fun to work with and play with and implement and .... etc)  is that there's an inherent simplicity, and much of the expressiveness and power stems from the freedom that simplicity brings. A simple system, requiring as few changes as possible, that will serve the purpose of internal run-time documentation, seems to me to be far superior to any over-engineered behemoth that requires changes to every part of the repl and compiler. Making something _extremely_ simple and then seeing how (or if) it evolves in community-useful ways may be a better path than trying to find all the ways that all of us could ever theoretically want to use it ... and it will be less limiting to the next generation who may come up with things that we never thought of.

I am not saying that a complex system is necessary bad or a simple one is necessarily good, just that it might be a good idea to follow a route that requires the least amount of future correction if it turns out not to be what we had hoped.

(I will be unavailable for the next few days due to Passover, so I'm sorry in advance for putting this point out at this time. I will try to catch up on the discussion on Saturday night my time. Only mentioning so you're all free to flame or disagree and not be worried that I got offended and left in a huff. :) )

-elf

The basic source of Scheme's simplicity is that it is built on top of a few basic principles and a few powerful building blocks through which everything else can be defined.  Two things immediately come to one's mind: iteration is handled through recursion and every tail call is essentially a goto and the hygienic macro system.

Documentation systems can become arbitrary complex and involved.  But that doesn't necessarily mean that the core of Scheme has to become much more complicated.  We just have to find out the missing simple concept that has to be added to the core of Scheme so that a user can build the documentation systems we are thinking about here as a library.

One such basic simple concept could be a facility to attach expand-time properties to identifiers like with Chez Scheme's `define-property'.  Such a concept could be both used for attaching data to identifiers as well as for adding static type information, etc.

Marc


 

>
> Best wishes,
> Arne
> --
> Unpolitisch sein
> heißt politisch sein
> ohne es zu merken
>
> --
> You received this message because you are subscribed to the Google Groups
> "scheme-reports-wg2" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to scheme-reports-...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/scheme-reports-wg2/87ftdgr1ai.fsf%40web.de
> .
>

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scheme-reports-wg2/CAEYrNrRZ0%3D8teAG%2BN3Xz7Hm3qyzNGzmsK8imWhGiS7ruemapkQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

Arthur A. Gleckler

unread,
Apr 9, 2020, 11:41:02 AM4/9/20
to scheme-re...@googlegroups.com
On Wed, Apr 8, 2020 at 11:40 PM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
 
While both are about documentation, I am not sure how much the concepts are related. In any case, it would be good if whatever solution we find for either concept it does not step on the toes of the other.

The unifying concept is making documentation available at runtime.

I agree about not stepping on toes.

This reminds me of debugging code.  Sometimes, you want to set a breakpoint whenever a particular closure is invoked.  Other times, you want to set a breakpoint whenever any closure bound to a specific variable is invoked.  At the implementation level, they're different concepts, but the user is trying to achieve just one thing: making sure that the debugger is invoked in the precisely correct place.
  
Through the closure value, the Scheme system could access the lambda form and thus any documentation attached to the code. Or do you really want to have different documentation on different closures of the same procedure?

Sorry, I misunderstood what you meant.  I'm thinking of an optional slot in each closure that points to the documentation in the same way that the closure points to its code.  All closures of the same lambda normally point to the same code object.  But maybe it would be better not to include a slot, and to use a weak hash table to map the code objects that are already present to the documentation string.  But that's an implementation detail.
 
The Racket editor's understanding of the source is very elaborate, I think.  It doesn't have to run the code, though. While at the REPL, this may be true, I don't think that you always want to run code to retrieve documentation.  Running general code has usually side effects.  Moreover, one run may differ from the next one.  We should also note that there are Scheme systems that are not REPL-centered but use the usual compile-and-run cycle.

Those Scheme systems might store the documentation strings in their binaries, if desired.

Marc Nieper-Wißkirchen

unread,
Apr 9, 2020, 12:06:04 PM4/9/20
to scheme-re...@googlegroups.com
Am Do., 9. Apr. 2020 um 17:41 Uhr schrieb Arthur A. Gleckler <a...@speechcode.com>:
On Wed, Apr 8, 2020 at 11:40 PM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
 
While both are about documentation, I am not sure how much the concepts are related. In any case, it would be good if whatever solution we find for either concept it does not step on the toes of the other.

The unifying concept is making documentation available at runtime.

I don't think that this is common to both concepts. Values exist at runtime, but bindings do not. Note that a compiler may remove and create any binding while processing Scheme code as long as the resulting code is correct.

Thus we really have two orthogonal concepts.
 
I agree about not stepping on toes.

This reminds me of debugging code.  Sometimes, you want to set a breakpoint whenever a particular closure is invoked.  Other times, you want to set a breakpoint whenever any closure bound to a specific variable is invoked.  At the implementation level, they're different concepts, but the user is trying to achieve just one thing: making sure that the debugger is invoked in the precisely correct place.

I'm trying to wrap my head around whether there are really two different concepts invoked in your example. Could you give some example code and describe the two debugging cases with this code?

 
  
Through the closure value, the Scheme system could access the lambda form and thus any documentation attached to the code. Or do you really want to have different documentation on different closures of the same procedure?

Sorry, I misunderstood what you meant.  I'm thinking of an optional slot in each closure that points to the documentation in the same way that the closure points to its code.  All closures of the same lambda normally point to the same code object.  But maybe it would be better not to include a slot, and to use a weak hash table to map the code objects that are already present to the documentation string.  But that's an implementation detail.

How can this concept of attaching information to runtime values *uniformly* extended to other runtime values?
 
 
The Racket editor's understanding of the source is very elaborate, I think.  It doesn't have to run the code, though. While at the REPL, this may be true, I don't think that you always want to run code to retrieve documentation.  Running general code has usually side effects.  Moreover, one run may differ from the next one.  We should also note that there are Scheme systems that are not REPL-centered but use the usual compile-and-run cycle.

Those Scheme systems might store the documentation strings in their binaries, if desired.

Documentation that is attached to values has to be stored in the binary, of course. Documentation attached to bindings behaves differently (see above).

Marc

 

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

Arthur A. Gleckler

unread,
Apr 9, 2020, 6:12:27 PM4/9/20
to scheme-re...@googlegroups.com
On Thu, Apr 9, 2020 at 9:06 AM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
 
The unifying concept is making documentation available at runtime.

I don't think that this is common to both concepts. Values exist at runtime, but bindings do not. Note that a compiler may remove and create any binding while processing Scheme code as long as the resulting code is correct.

In any context in which someone can refer to a variable by name, the system should be able to retrieve documentation on that variable.

Frankly, however, I'm most interested in a proposal that covers bindings of variables to procedures and syntax, where a mapping between the values and the documentation is what is maintained.  I'd be happy to forego anything else in the short term following the Pareto principle.
  
This reminds me of debugging code.  Sometimes, you want to set a breakpoint whenever a particular closure is invoked.  Other times, you want to set a breakpoint whenever any closure bound to a specific variable is invoked.  At the implementation level, they're different concepts, but the user is trying to achieve just one thing: making sure that the debugger is invoked in the precisely correct place.

I'm trying to wrap my head around whether there are really two different concepts invoked in your example. Could you give some example code and describe the two debugging cases with this code?

In this example, it would be sufficient to set a breakpoint with reference to either the variable factorial or the value that it holds:

On the other hand, if I'm debugging find-in-first-ten below, I'd like to set the breakpoint with reference to the variable predicate, not to the procedure, odd?, to which it is bound:

Sorry, I misunderstood what you meant.  I'm thinking of an optional slot in each closure that points to the documentation in the same way that the closure points to its code.  All closures of the same lambda normally point to the same code object.  But maybe it would be better not to include a slot, and to use a weak hash table to map the code objects that are already present to the documentation string.  But that's an implementation detail.

How can this concept of attaching information to runtime values *uniformly* extended to other runtime values?

Doesn't a weak hash table solve that problem?  

Marc Nieper-Wißkirchen

unread,
Apr 10, 2020, 4:39:14 AM4/10/20
to scheme-re...@googlegroups.com
Am Fr., 10. Apr. 2020 um 00:12 Uhr schrieb Arthur A. Gleckler <a...@speechcode.com>:
On Thu, Apr 9, 2020 at 9:06 AM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
 
The unifying concept is making documentation available at runtime.

I don't think that this is common to both concepts. Values exist at runtime, but bindings do not. Note that a compiler may remove and create any binding while processing Scheme code as long as the resulting code is correct.

In any context in which someone can refer to a variable by name, the system should be able to retrieve documentation on that variable.

Frankly, however, I'm most interested in a proposal that covers bindings of variables to procedures and syntax, where a mapping between the values and the documentation is what is maintained.  I'd be happy to forego anything else in the short term following the Pareto principle.
  
This reminds me of debugging code.  Sometimes, you want to set a breakpoint whenever a particular closure is invoked.  Other times, you want to set a breakpoint whenever any closure bound to a specific variable is invoked.  At the implementation level, they're different concepts, but the user is trying to achieve just one thing: making sure that the debugger is invoked in the precisely correct place.

I'm trying to wrap my head around whether there are really two different concepts invoked in your example. Could you give some example code and describe the two debugging cases with this code?

In this example, it would be sufficient to set a breakpoint with reference to either the variable factorial or the value that it holds:

On the other hand, if I'm debugging find-in-first-ten below, I'd like to set the breakpoint with reference to the variable predicate, not to the procedure, odd?, to which it is bound:

Thanks for the example.  So in both cases, the breakpoints can be associated with the variables (vs with the values) and you get what you want.

But what is an example where you really need to associate the breakpoint with a value?

Sorry, I misunderstood what you meant.  I'm thinking of an optional slot in each closure that points to the documentation in the same way that the closure points to its code.  All closures of the same lambda normally point to the same code object.  But maybe it would be better not to include a slot, and to use a weak hash table to map the code objects that are already present to the documentation string.  But that's an implementation detail.

How can this concept of attaching information to runtime values *uniformly* extended to other runtime values?

Doesn't a weak hash table solve that problem?  

No, not uniformly for arbitrary values.  A weak hash-table would associate the documentation to an individual closure, not to the underlying code.

Marc

 

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

Alaric Snell-Pym

unread,
Apr 10, 2020, 8:02:03 AM4/10/20
to scheme-re...@googlegroups.com
On 07/04/2020 23:22, Arne Babenhauserheide wrote:
>
> You might be interested in the doctest stuff I did:
>
> (define (A)
> "returns 'A"
> #((tests (test-eqv 'A (A))
> (test-assert #t)))
> 'A)
>
> This already works in Guile and is quite similar to your ? idea, though
> not as general.

Nice!

> '?' as name sounds dangerous to me, though: This is already used in
> other places (it’s far too common). Basically every special character
> for not-absolutely-core-operations like λ and '`, is dangerous, because
> it increases complexity for human readers and newcomers.

Yeah, it was just a throwaway suggestion, to have some simple short
syntax for inline comments :-) It's hard to beat ";"...

ABS
signature.asc

Arthur A. Gleckler

unread,
Apr 10, 2020, 11:26:23 AM4/10/20
to scheme-re...@googlegroups.com
On Fri, Apr 10, 2020 at 1:39 AM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
 
Thanks for the example.  So in both cases, the breakpoints can be associated with the variables (vs with the values) and you get what you want.

But what is an example where you really need to associate the breakpoint with a value?
How can this concept of attaching information to runtime values *uniformly* extended to other runtime values?

Doesn't a weak hash table solve that problem?  

No, not uniformly for arbitrary values.  A weak hash-table would associate the documentation to an individual closure, not to the underlying code.

That's why you need both concepts.

Marc Nieper-Wißkirchen

unread,
Apr 10, 2020, 11:39:09 AM4/10/20
to scheme-re...@googlegroups.com
Am Fr., 10. Apr. 2020 um 17:26 Uhr schrieb Arthur A. Gleckler <a...@speechcode.com>:
On Fri, Apr 10, 2020 at 1:39 AM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
 
Thanks for the example.  So in both cases, the breakpoints can be associated with the variables (vs with the values) and you get what you want.

But what is an example where you really need to associate the breakpoint with a value?

Here, I would associate the breakpoint with the binding of `log'.  Of course, we would have to say what this actually means. It would mean that, say, `(define (log parameters) ...` is expanded to `(define log (debugger-maybe-break-here (lambda (parameters) ...)))'

It has to happen at expand/compile time because bindings are no runtime concept (except for bindings at the REPL).

How can this concept of attaching information to runtime values *uniformly* extended to other runtime values?

Doesn't a weak hash table solve that problem?  

No, not uniformly for arbitrary values.  A weak hash-table would associate the documentation to an individual closure, not to the underlying code.

That's why you need both concepts.

That would raise the concept count to three: documentation associated with values (distinguished by eq? or eqv?), documentation associated with bindings, and documentation associated with particular runtime values (procedures).

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

Arne Babenhauserheide

unread,
Apr 10, 2020, 1:33:26 PM4/10/20
to scheme-re...@googlegroups.com

Arthur A. Gleckler <a...@speechcode.com> writes:
>> I don't think that this is common to both concepts. Values exist at
>> runtime, but bindings do not. Note that a compiler may remove and create
>> any binding while processing Scheme code as long as the resulting code is
>> correct.
>>
>
> In any context in which someone can refer to a variable by name, the system
> should be able to retrieve documentation on that variable.

How does this work for this?

(define (foo) "doc" #t)
(define bar foo)

I would expect

(procedure-documentation bar)

to retrieve the documentation of foo ("doc").

Arthur A. Gleckler

unread,
Apr 10, 2020, 2:12:54 PM4/10/20
to scheme-re...@googlegroups.com
On Fri, Apr 10, 2020 at 8:39 AM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
 
Here, I would associate the breakpoint with the binding of `log'.  Of course, we would have to say what this actually means. It would mean that, say, `(define (log parameters) ...` is expanded to `(define log (debugger-maybe-break-here (lambda (parameters) ...)))'

I don't see how that's different than associating the breakpoint with the procedure value itself.
 
It has to happen at expand/compile time because bindings are no runtime concept (except for bindings at the REPL).

That means that you have to re-expand/compile the code in order to set a breakpoint.  That's a common approach, but makes for a much less pleasant debugging experience than using debuggers in other languages, even C.
  
That would raise the concept count to three: documentation associated with values (distinguished by eq? or eqv?), documentation associated with bindings, and documentation associated with particular runtime values (procedures).
 
I feel like we're going around in circles.

I'll say again that I'd be happy with just the ability to associate documentation with procedures and syntax transformers.  That's all I was thinking when I started this thread.  Other folks asked about associating documentation with bindings or with arbitrary objects, and that would be nice, but I'd like to start with a simple, straightforward proposal.

Unfortunately, as SRFI editor, I don't feel comfortable submitting SRFIs myself.  It seems like a conflict of interest, especially if I'm the lead author as opposed to just a contributor.  But if someone else finds this basic idea useful, I would love to see a proposal.

Arthur A. Gleckler

unread,
Apr 10, 2020, 2:23:52 PM4/10/20
to scheme-re...@googlegroups.com
On Fri, Apr 10, 2020 at 10:33 AM Arne Babenhauserheide <arne...@web.de> wrote:
 
> In any context in which someone can refer to a variable by name, the system
> should be able to retrieve documentation on that variable.

How does this work for this?

    (define (foo) "doc" #t)
    (define bar foo)

I would expect

    (procedure-documentation bar)

to retrieve the documentation of foo ("doc"). 

Yes, that's all I would hope for.  The idea of associating documentation with variables themselves is interesting, but less important.  I'm not going to push for that.

I responded to John's questions above about expanding the scope of documentation strings saying that I would support having them on variable bindings, not just values, and on non-procedure, non-syntax-transformer values in addition to bindings.  However, there doesn't seem to be any clear proposal for how to do that, whereas adding doc strings to procedures and syntax transformers is straightforward, so I withdraw support for the expanded ideas.

Marc Nieper-Wißkirchen

unread,
Apr 10, 2020, 3:36:11 PM4/10/20
to scheme-re...@googlegroups.com
Am Fr., 10. Apr. 2020 um 20:12 Uhr schrieb Arthur A. Gleckler <a...@speechcode.com>:
On Fri, Apr 10, 2020 at 8:39 AM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
 
Here, I would associate the breakpoint with the binding of `log'.  Of course, we would have to say what this actually means. It would mean that, say, `(define (log parameters) ...` is expanded to `(define log (debugger-maybe-break-here (lambda (parameters) ...)))'

I don't see how that's different than associating the breakpoint with the procedure value itself.
 
It has to happen at expand/compile time because bindings are no runtime concept (except for bindings at the REPL).

That means that you have to re-expand/compile the code in order to set a breakpoint.  That's a common approach, but makes for a much less pleasant debugging experience than using debuggers in other languages, even C.

The problem is that the compiler may have optimized away a lot of code or that log may have been inlined in the procedures so that there is not much to attach to at runtime.
 
  
That would raise the concept count to three: documentation associated with values (distinguished by eq? or eqv?), documentation associated with bindings, and documentation associated with particular runtime values (procedures).
 
I feel like we're going around in circles.

I'll say again that I'd be happy with just the ability to associate documentation with procedures and syntax transformers.  That's all I was thinking when I started this thread.  Other folks asked about associating documentation with bindings or with arbitrary objects, and that would be nice, but I'd like to start with a simple, straightforward proposal.

Unfortunately, as SRFI editor, I don't feel comfortable submitting SRFIs myself.  It seems like a conflict of interest, especially if I'm the lead author as opposed to just a contributor.  But if someone else finds this basic idea useful, I would love to see a proposal.

I can submit a SRFI for attaching meta-information like documentation to procedures and syntax transformers, maybe after we have discussed it a bit if you like.

I do think the idea to attach such information to procedures and to syntax transformers (and maybe record (types)) is useful so I am willing to help here. As I said, the only thing I do not want is to step on the toes of the orthogonal thing to attach properties to bindings.

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

Marc Nieper-Wißkirchen

unread,
Apr 10, 2020, 3:41:08 PM4/10/20
to scheme-re...@googlegroups.com
Am Fr., 10. Apr. 2020 um 19:33 Uhr schrieb Arne Babenhauserheide <arne...@web.de>:

Arthur A. Gleckler <a...@speechcode.com> writes:
>> I don't think that this is common to both concepts. Values exist at
>> runtime, but bindings do not. Note that a compiler may remove and create
>> any binding while processing Scheme code as long as the resulting code is
>> correct.
>>
>
> In any context in which someone can refer to a variable by name, the system
> should be able to retrieve documentation on that variable.

How does this work for this?

    (define (foo) "doc" #t)
    (define bar foo)

I would expect

    (procedure-documentation bar)

to retrieve the documentation of foo ("doc").

Not necessarily, I would say, as in the following example:

(define (make-address name street city) (list name street city))
(define address-name car)
(define address-street cadr)
(define address-city caddr)

In this case, we would want to attach other documentation to the three accessors than that is attached to caXr.

NB: Chez Scheme has `alias', which associates a new identifier with exactly the same binding than a previous one.  In the above example, changing `(define bar foo)' with `(alias bar foo)' would yield the "expected".

Marc
 

Best wishes,
Arne
--
Unpolitisch sein
heißt politisch sein
ohne es zu merken

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

Alaric Snell-Pym

unread,
Apr 10, 2020, 3:51:09 PM4/10/20
to scheme-re...@googlegroups.com
On 10/04/2020 19:12, Arthur A. Gleckler wrote:
> On Fri, Apr 10, 2020 at 8:39 AM Marc Nieper-Wißkirchen <
> marc....@gmail.com> wrote:
>
>
>> Here, I would associate the breakpoint with the binding of `log'. Of
>> course, we would have to say what this actually means. It would mean that,
>> say, `(define (log parameters) ...` is expanded to `(define log
>> (debugger-maybe-break-here (lambda (parameters) ...)))'
>>
>
> I don't see how that's different than associating the breakpoint with the
> procedure value itself.

One difference could be what would one expects to happen if (set! log
...something...) happens, although mutating global procedure bindings is
generally inviting confusion anyway :-)

>
>> It has to happen at expand/compile time because bindings are no runtime
>> concept (except for bindings at the REPL).
>>
>
> That means that you have to re-expand/compile the code in order to set a
> breakpoint. That's a common approach, but makes for a much less pleasant
> debugging experience than using debuggers in other languages, even C.
>

I think another aspect in this "what are bindings at runtime?" tangent
is that the kinds of smart editor / documentation generation thing
people often imagine docstrings being used in will probably be working
with some kind of runtime representation of modules. Sure, *repls* will
have values bound in an environment and we'd hope to have some procedure
that could return the docs of an arbitrary object, and invoke it by
referencing bindings such as + or call-with-current-continuation; but an
"offline" tool such as an editor or documentation generator will have
some kind of interface to Scheme library definitions and work from that.
Currently such things largely seem to work by being fed the source files
themselves and loading them from the filesystem, which is why approaches
based around special comments are popular, but it might be nice to think
about a way of querying the list of available modules in Scheme and
accessing their lists of bindings and attached documentation or other
metadata, so that such tools can work with separately-compiled modules,
extract documentation that's generated by macro expansions, etc?
signature.asc

Arthur A. Gleckler

unread,
Apr 10, 2020, 4:14:21 PM4/10/20
to scheme-re...@googlegroups.com
On Fri, Apr 10, 2020 at 12:36 PM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
 
The problem is that the compiler may have optimized away a lot of code or that log may have been inlined in the procedures so that there is not much to attach to at runtime.

A least-common-denominator debugger and runtime system will have this problem, and for that reason, it's hard to standardize debugging tools.  But modern compilers for other languages like JavaScript and Smalltalk can de-optimize at runtime for the purpose of the debugger and other tools.  I'd love to see that kind of capability in Scheme some day.
  
I can submit a SRFI for attaching meta-information like documentation to procedures and syntax transformers, maybe after we have discussed it a bit if you like.

I do think the idea to attach such information to procedures and to syntax transformers (and maybe record (types)) is useful so I am willing to help here. As I said, the only thing I do not want is to step on the toes of the orthogonal thing to attach properties to bindings.
 
That would be wonderful.  Thanks.

Marc Nieper-Wißkirchen

unread,
Apr 10, 2020, 4:25:58 PM4/10/20
to scheme-re...@googlegroups.com
Am Fr., 10. Apr. 2020 um 22:14 Uhr schrieb Arthur A. Gleckler <a...@speechcode.com>:
On Fri, Apr 10, 2020 at 12:36 PM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
 
The problem is that the compiler may have optimized away a lot of code or that log may have been inlined in the procedures so that there is not much to attach to at runtime.

A least-common-denominator debugger and runtime system will have this problem, and for that reason, it's hard to standardize debugging tools.  But modern compilers for other languages like JavaScript and Smalltalk can de-optimize at runtime for the purpose of the debugger and other tools.  I'd love to see that kind of capability in Scheme some day.

I agree that we shouldn't try to standardize debugging tools. But we can standardize some tools that help debugging. For example, SRFI 157 reifies the concept of the call stack.

As for the other thing, expanding and compiling Scheme code can be very fast so it is probably not that bad that code has to be recompiled when a breakpoint or some other introspection facility is added.

  
I can submit a SRFI for attaching meta-information like documentation to procedures and syntax transformers, maybe after we have discussed it a bit if you like.

I do think the idea to attach such information to procedures and to syntax transformers (and maybe record (types)) is useful so I am willing to help here. As I said, the only thing I do not want is to step on the toes of the orthogonal thing to attach properties to bindings.
 
That would be wonderful.  Thanks.

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

Arthur A. Gleckler

unread,
Apr 10, 2020, 4:31:58 PM4/10/20
to scheme-re...@googlegroups.com
On Fri, Apr 10, 2020 at 1:25 PM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
 
As for the other thing, expanding and compiling Scheme code can be very fast so it is probably not that bad that code has to be recompiled when a breakpoint or some other introspection facility is added.
 
The problem happens when the procedure is closed over some bindings.  Recompiling is likely to lose those bindings.

Marc Nieper-Wißkirchen

unread,
Apr 10, 2020, 4:38:26 PM4/10/20
to scheme-re...@googlegroups.com
Yes, this would be a problem in running code. But without recompiling, we may miss some invocations we would like to trap because the compiler has chosen to inline the procedure at a few places.  If we want to make sure that no recompilation is needed, the whole structure of the program has to be preserved.  I am wondering whether this means that we have to deal with the fexpr problem (https://en.wikipedia.org/wiki/Fexpr#cite_note-Wa98p189-10) in that no source-to-source rewriting can be made at all.
 
--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

Arthur A. Gleckler

unread,
Apr 10, 2020, 4:49:16 PM4/10/20
to scheme-re...@googlegroups.com
On Fri, Apr 10, 2020 at 1:38 PM Marc Nieper-Wißkirchen <marc....@gmail.com> wrote:
 
Yes, this would be a problem in running code. But without recompiling, we may miss some invocations we would like to trap because the compiler has chosen to inline the procedure at a few places.  If we want to make sure that no recompilation is needed, the whole structure of the program has to be preserved.  I am wondering whether this means that we have to deal with the fexpr problem (https://en.wikipedia.org/wiki/Fexpr#cite_note-Wa98p189-10) in that no source-to-source rewriting can be made at all.
 
Ideally, the compiler would have a record of what places it had done inlining.  As I understand it, the Hotspot VM for Java does similar things for deoptimizing.  This would definitely not be for a lightweight Scheme, though.

Arne Babenhauserheide

unread,
Apr 11, 2020, 4:23:06 AM4/11/20
to scheme-re...@googlegroups.com
This looks like a case for define with docstring — which would then
override the original:

(define (make-address name street city) "Create a street address object" (list name street city))
(define address-name "retrieve the person name of the address" car)
(define address-street "retrieve the street name of the address" cadr)
(define address-city "retrieve the city of the address" caddr)

In the default case I would still expect the docstring to be preserved.

Marc Nieper-Wißkirchen

unread,
Apr 11, 2020, 4:40:08 AM4/11/20
to scheme-re...@googlegroups.com
This would mean that including a docstring would change the behavior of the program (if documentation is attached to values, not bindings):

(define address-name car)
(eqv? address-name car) => #t

(define address-name "doc" car)
(eqv? address-name car) => #f
That's not good. So for that particular use case, documentation associated with bindings and not values seems superior.

Marc


Best wishes,
Arne
--
Unpolitisch sein
heißt politisch sein
ohne es zu merken

--
You received this message because you are subscribed to the Google Groups "scheme-reports-wg2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scheme-reports-...@googlegroups.com.

Arne Babenhauserheide

unread,
Apr 11, 2020, 5:44:22 AM4/11/20
to scheme-re...@googlegroups.com

Marc Nieper-Wißkirchen <marc....@gmail.com> writes:

> Am Sa., 11. Apr. 2020 um 10:23 Uhr schrieb Arne Babenhauserheide <
>> >> > In any context in which someone can refer to a variable by name, the
>> >> > system should be able to retrieve documentation on that variable.
>> >>
>> >> How does this work for this?
>> >>
>> >> (define (foo) "doc" #t)
>> >> (define bar foo)
>> >>
>> >> I would expect
>> >>
>> >> (procedure-documentation bar)
>> >>
>> >> to retrieve the documentation of foo ("doc").
>> >
>> > Not necessarily, I would say, as in the following example:
>> >
>> > (define (make-address name street city) (list name street city))
>> > (define address-name car)
>> >
>> This looks like a case for define with docstring — which would then
>> override the original:
>>
>> (define (make-address name street city) "Create a street address object"
>> (list name street city))
>> (define address-name "retrieve the person name of the address" car)
>>
>> In the default case I would still expect the docstring to be preserved.
>
> (define address-name car)
> (eqv? address-name car) => #t
>
> (define address-name "doc" car)
> (eqv? address-name car) => #f
> That's not good. So for that particular use case, documentation associated
> with bindings and not values seems superior.

Yes, it looks to me like that, too.

An option would be to have define preserve/copy the docstring of what’s referenced.

Arne Babenhauserheide

unread,
Apr 26, 2020, 4:03:19 AM4/26/20
to scheme-re...@googlegroups.com, Alaric Snell-Pym
Yes, it is hard. Terribly hard. In my presentation of wisp I showed,
just how hard it is:

> The most common non-letter non-math characters in prose are, as gathered
> from a corpus I use to optimize keyboard layouts:
>
> .,":'_#?!;()

Note that neither {} nor [] are in there. Round parentheses are more
than 20 times more common to prose than any other type of paired
identifiers. Here’s a occurrence-count, weighted by source (so not an
exact match to just counting letters):

2039345 (
74007 <
70549 {
68612 &
31498 [

Source: https://hg.sr.ht/~arnebab/evolve-keyboard-layout/browse/1-gramme.15.txt?rev=048cfd220458f7a38009395f9f52c6d4b9fa1ac0


For wisp I use : to start inline parentheses, because that absolutely
requires special syntax. If I could have avoided that, I would have. It
clashes with type-definitions used in typed racket.


Best wishes,
Arne
--
Unpolitisch sein
heißt politisch sein
ohne es zu merken
- https://www.draketo.de
Reply all
Reply to author
Forward
0 new messages