Symbols
Oliver Bandel
Hi,
how will symbols be created and handled in scheme?
Are symbols normally created in scanning/parsing of data?
How will they be used in other ways (not parsing, but
creating)?
Where can I find some minimal example-sources for parsing
and usage of symbols?
TIA,
Oliver
Stephan H.M.J. Houben
>3. by being generated by the symbol->string procedure
Of course I mean the string->symbol procedure.
Stephan
Stephan H.M.J. Houben
>
>Hi,
>
>how will symbols be created and handled in scheme?
From R5RS:
"Symbols are objects whose usefulness rests on the fact that two symbols are
identical (in the sense of eqv?) if and only if their names
are spelled the same way. This is exactly the property needed to represent
identifiers in programs, and so most implementations of
Scheme use them internally for that purpose. Symbols are useful for many other
applications; for instance, they may be used the way
enumerated values are used in Pascal."
Symbols can in R5RS only be created in 3 ways:
1. by being explicitely quoted in the progrtam source
2. by being generated by the read procedure
3. by being generated by the symbol->string procedure
Implementations have to ensure eq?-ness of symbols, which is commonly
done by maintaining a table of all symbols that have already be generated.
(This technique is called "interning".) Often, the table is a "weak" table,
so that unused symbols can be garbage collected.
>Are symbols normally created in scanning/parsing of data?
>How will they be used in other ways (not parsing, but
>creating)?
See above.
>Where can I find some minimal example-sources for parsing
>and usage of symbols?
In R5RS:
(eq? 'mISSISSIppi 'mississippi)
===> #t
(string->symbol "mISSISSIppi")
===> the symbol with name "mISSISSIppi"
(eq? 'bitBlt (string->symbol "bitBlt"))
===> #f
(eq? 'JollyWog
(string->symbol
(symbol->string 'JollyWog)))
===> #t
(string=? "K. Harper, M.D."
(symbol->string
(string->symbol "K. Harper, M.D.")))
===> #t
Stephan
Ray Dillinger
"Stephan H.M.J. Houben" wrote:
>
> From R5RS:
> In R5RS:
>
> (eq? 'mISSISSIppi 'mississippi)
> ===> #t
> (string->symbol "mISSISSIppi")
> ===> the symbol with name "mISSISSIppi"
> (eq? 'bitBlt (string->symbol "bitBlt"))
> ===> #f
> (eq? 'JollyWog
> (string->symbol
> (symbol->string 'JollyWog)))
> ===> #t
> (string=? "K. Harper, M.D."
> (symbol->string
> (string->symbol "K. Harper, M.D.")))
> ===> #t
>
Does R5RS make any prescriptions about free uses of symbols
created with string->symbol? For example, if I say
(define (string->symbol "foo ") (lambda (a b) (+ a b)))
(define foo (lambda (a b)(- a b)))
is (foo a b) with a numeric a and a numeric non-zero b required
to evaluate to a different number than (foo a b)?
It seems to me that R5RS requires that
(foo a b) evaluate differently from (((string->symbol "foo ")) a b)
but doesn't say anything at all about a free use of a symbol that
includes something that would otherwise be a symbol delimiter.
Would it be legal for the symbol created by string->symbol to use
escape codes followed by numbers for characters that might cause
the parser trouble otherwise (notably spaces and parens)? For
example if
(eqv? (string->symbol "foo ") foo\20 ) ==> #t
does it violate the semantics of R5RS?
Note that allowing symbols to include a backslash would be a
language extension not described in R5RS, so using backslash as
an escape character would effectively create a separate namespace
for string->symbol on args with troublesome characters, guaranteed
not to collide with strictly R5RS symbols. So, I argue that this
does not pre-empt a language construct defined by the standard and
ought to be legal.
I hope it's legal anyway; it would make writing parsers a lot
easier.
Bear
Stephan H.M.J. Houben
>Does R5RS make any prescriptions about free uses of symbols
>created with string->symbol?
What do you mean with "free uses"?
>For example, if I say
>
>(define (string->symbol "foo ") (lambda (a b) (+ a b)))
This is not legal Scheme. What you want is something like:
(eval `(define ,(string->symbol "foo") (lambda (a b) (+ a b)))
(interaction-environment))
Note that this is not required to work in an R5RS Scheme; implementations
"may" allow definitions in eval, and interaction-environment is optional.
Also note that there is a gotcha with the case of the symbol:
(string->symbol "foo") always creates the symbol foo in lowercase,
but foo as read by the Scheme reader might create an uppercase symbol FOO.
In which case the variable named by the symbol foo will still be bound
to the procedure, but you cannot easily access it.
But e.g. in DrScheme everything works fine (DrScheme supports the extensions
to eval, and reads lowercase symbols.)
>(define foo (lambda (a b)(- a b)))
This is what the procedure eval actually sees.
>is (foo a b) with a numeric a and a numeric non-zero b required
>to evaluate to a different number than (foo a b)?
No. Why would it?
>It seems to me that R5RS requires that
>(foo a b) evaluate differently from (((string->symbol "foo ")) a b)
Sure, the second is an error. You try to apply a symbol. BTW, I think you
meant one pair of parentheses less around (string->symbol "foo"), but
that doesn't change the situation in this case.
>but doesn't say anything at all about a free use of a symbol that
>includes something that would otherwise be a symbol delimiter.
>
>Would it be legal for the symbol created by string->symbol to use
>escape codes followed by numbers for characters that might cause
>the parser trouble otherwise (notably spaces and parens)?
Sure, any string is legal input to string->symbol. Not every symbol
that can be generated by string->symbol can appear as a literal
in the program text.
>For
>example if
>
>(eqv? (string->symbol "foo ") foo\20 ) ==> #t
>
>does it violate the semantics of R5RS?
Yes, since \ may not appear in identifiers. But note that
(eqv? (string->symbol "foo") foo)
compares (string->symbol "foo") with the value bound to foo, NOT
to the symbol foo.
You appear to be confusing symbols with variables. Don't do that.
This is Scheme, not Lisp.
>Note that allowing symbols to include a backslash would be a
>language extension not described in R5RS, so using backslash as
>an escape character would effectively create a separate namespace
>for string->symbol on args with troublesome characters, guaranteed
>not to collide with strictly R5RS symbols. So, I argue that this
>does not pre-empt a language construct defined by the standard and
>ought to be legal.
??? Because it is not in R5RS, it ought to be legal?
That doesn't compute.
>I hope it's legal anyway; it would make writing parsers a lot
>easier.
It is an extension of some Schemes. Why would it make writing parsers
easier?
Stephan
Ray Dillinger
>
> In article <3C352512...@sonic.net>, Ray Dillinger wrote:
> >Does R5RS make any prescriptions about free uses of symbols
> >created with string->symbol?
>
> What do you mean with "free uses"?
>
> >For example, if I say
> >
> >(define (string->symbol "foo ") (lambda (a b) (+ a b)))
>
> This is not legal Scheme. What you want is something like:
>
> (eval `(define ,(string->symbol "foo") (lambda (a b) (+ a b)))
> (interaction-environment))
you know, I've done that trick a dozen times and never
discovered a place where it was illegal. So I went back
to R5RS, and sure enough, you're right. (define) is not
required to evaluate its first argument. Imagine
my surprise. However, it took a close reading of
the Denotational Semantics to discover that. Good catch!
However, your rewrite completely misses the point of my
question, which was about defining a variable whose name
contains a space or a paren. A version of your rewrite
that captures the case I'm asking about would be something
like
(eval '(define ,(string->symbol "foo ") (lambda (a b) (+ a b)))
(interaction-environment)
> Note that this is not required to work in an R5RS Scheme;
Duly noted.
> ...gotcha about case....
Duly noted.
> >(define foo (lambda (a b)(- a b)))
>
> This is what the procedure eval actually sees.
No, it sees (define ##### (lambda (a b) (- a b)))
where ##### is the symbol created by string->symbol,
whose name is spelled f, o, o, space.
> >is (foo a b) with a numeric a and a numeric non-zero b required
> >to evaluate to a different number than (foo a b)?
>
> No. Why would it?
Because under some readings of the denotational semantics,
(foo a b) (where only one space separates "foo" from "a")
refers to a variable whose name is spelled f, o, o
and (foo a b) (where two spaces separate "foo" from "a")
refers to a variable whose name is spelled f, o, o, space.
> >It seems to me that R5RS requires that
> >(foo a b) evaluate differently from (((string->symbol "foo ")) a b)
>
> Sure, the second is an error. You try to apply a symbol. BTW, I think you
> meant one pair of parentheses less around (string->symbol "foo"), but
> that doesn't change the situation in this case.
Uh, yes it does. Extra parens trigger an evaluation, and if
it's a variable name, gets you the value.
> >Would it be legal for the symbol created by string->symbol to use
> >escape codes followed by numbers for characters that might cause
> >the parser trouble otherwise (notably spaces and parens)?
>
> Sure, any string is legal input to string->symbol. Not every symbol
> that can be generated by string->symbol can appear as a literal
> in the program text.
>
> >For
> >example if
> >
> >(eqv? (string->symbol "foo ") foo\20 ) ==> #t
> >
> >does it violate the semantics of R5RS?
>
> Yes, since \ may not appear in identifiers.
R5RS does not require that an implementation allow \ to appear
in identifiers, but it doesn't disallow it either.
> >Note that allowing symbols to include a backslash would be a
> >language extension not described in R5RS, so using backslash as
> >an escape character would effectively create a separate namespace
> >for string->symbol on args with troublesome characters, guaranteed
> >not to collide with strictly R5RS symbols. So, I argue that this
> >does not pre-empt a language construct defined by the standard and
> >ought to be legal.
> ??? Because it is not in R5RS, it ought to be legal?
> That doesn't compute.
Because R5RS doesn't prohibit it, and because it will not
change the semantics of any R5RS-legal programs, I argue
that it ought to be legal. IOW, if your implementation
allows additional namespace in the symbols, you have the
right to use it as you please.
> >I hope it's legal anyway; it would make writing parsers a lot
> >easier.
>
> It is an extension of some Schemes. Why would it make writing parsers
> easier?
because then the parser wouldn't have to be redefinable at
runtime to look out for spaces or parens that, because of
strange definitions in play, turn out not to be delimiters.
Bear
Jeffrey Siegal
> you know, I've done that trick a dozen times and never
> discovered a place where it was illegal.
What trick?
(define (string->symbol "foo") ...) ??
In which Scheme implementation does that work?
Feuer
Jeffrey Siegal wrote:
> Ray Dillinger wrote:
> (define (string->symbol "foo") ...) ??
>
> In which Scheme implementation does that work?
All of them :-). It defines the function string->symbol.
Feuer
Ray Dillinger wrote:
>Because under some readings of the denotational semantics,
> (foo a b) (where only one space separates "foo" from "a")
> refers to a variable whose name is spelled f, o, o
> and (foo a b) (where two spaces separate "foo" from "a")
> refers to a variable whose name is spelled f, o, o, space.
Umm... I think the standard says that a symbol (read by the parser/lexer) does
not contain spaces, and that extra spaces are ignored outside of string
literals...
> > >It seems to me that R5RS requires that
> > >(foo a b) evaluate differently from (((string->symbol "foo ")) a b)
> >
> > Sure, the second is an error. You try to apply a symbol. BTW, I think you
> > meant one pair of parentheses less around (string->symbol "foo"), but
> > that doesn't change the situation in this case.
>
> Uh, yes it does. Extra parens trigger an evaluation, and if
> it's a variable name, gets you the value.
No they don't. They trigger an application. You can't apply a name, you can
only apply a function.
(foo a b) may work, but (('foo) a b) most certainly will not.
David Feuer
ol...@pobox.com
> Does R5RS make any prescriptions about free uses of symbols
> created with string->symbol? For example, if I say
> (define (string->symbol "foo ") (lambda (a b) (+ a b)))
> (define foo (lambda (a b)(- a b)))
> is (foo a b) with a numeric a and a numeric non-zero b required
> to evaluate to a different number than (foo a b)?
What you wish can be done:
http://groups.google.com/groups?selm=7eb8ac3e.0111051503.d8cf750%40posting.google.com
The end of that article shows how to use identifiers with embedded
spaces, and even how to use an empty string and "numbers" as
identifiers.
As the article says, this forces us to draw a distinction between
notation and denotation.
Jeffrey Siegal
No it doesn't because "foo" is not a valid argument name.
Sander Vesik
> "Stephan H.M.J. Houben" wrote:
>>
>> In article <3C352512...@sonic.net>, Ray Dillinger wrote:
>> >Does R5RS make any prescriptions about free uses of symbols
>> >created with string->symbol?
>>
>> What do you mean with "free uses"?
>>
>> >For example, if I say
>> >
>> >(define (string->symbol "foo ") (lambda (a b) (+ a b)))
>>
>> This is not legal Scheme. What you want is something like:
>>
>> (eval `(define ,(string->symbol "foo") (lambda (a b) (+ a b)))
>> (interaction-environment))
> you know, I've done that trick a dozen times and never
> discovered a place where it was illegal. So I went back
> to R5RS, and sure enough, you're right. (define) is not
> required to evaluate its first argument. Imagine
> my surprise. However, it took a close reading of
> the Denotational Semantics to discover that. Good catch!
<plug>you haven't been using scheme48, then</plug>
I tried this and several other forms (inc. in macros to generate
interesting things) but none of them worked. then I re-read the
standard....
> However, your rewrite completely misses the point of my
> question, which was about defining a variable whose name
> contains a space or a paren. A version of your rewrite
> that captures the case I'm asking about would be something
> like
> (eval '(define ,(string->symbol "foo ") (lambda (a b) (+ a b)))
> (interaction-environment)
I don't think this would work or be legal. define does not expect
*anything* but a literal name - not a symbol...
>> >(define foo (lambda (a b)(- a b)))
>>
>> This is what the procedure eval actually sees.
>
> No, it sees (define ##### (lambda (a b) (- a b)))
> where ##### is the symbol created by string->symbol,
> whose name is spelled f, o, o, space.
But does it expect a symbol?
> Uh, yes it does. Extra parens trigger an evaluation, and if
> it's a variable name, gets you the value.
But it is not a variable name, its a symbol 8-)
[snip]
> Bear
--
Sander
+++ Out of cheese error +++
Lauri Alanko
>> (eval '(define ,(string->symbol "foo ") (lambda (a b) (+ a b)))
>> (interaction-environment)
>
> I don't think this would work or be legal. define does not expect
> *anything* but a literal name - not a symbol...
>> No, it sees (define ##### (lambda (a b) (- a b)))
>> where ##### is the symbol created by string->symbol,
>> whose name is spelled f, o, o, space.
>
> But does it expect a symbol?
>> Uh, yes it does. Extra parens trigger an evaluation, and if
>> it's a variable name, gets you the value.
>
> But it is not a variable name, its a symbol 8-)
Point made. A symbol indeed is not an identifier. But part of the magic of
eval is that it turns symbols into identifiers, strings into string
literals, lists into applications and other syntactic forms, etc. It is, in
a way, a reverse of quote: quote turns an expression into a (literal)
object, whereas eval turns an object into an expression, and consequently
evaluates that.
It would perhaps better distinguish the difference between expressions and
objects, if eval used a string as an input, instead of a structured object.
But the latter allows for easier syntax manipulation.
Of course, a syntax-case -style interface for syntax manipulation makes much
clearer the difference between expressions and objects, even though both
have the same kind of a printed representation...
Lauri Alanko
l...@iki.fi
Barry Margolin
Ray Dillinger <be...@sonic.net> wrote:
>you know, I've done that trick a dozen times and never
>discovered a place where it was illegal. So I went back
>to R5RS, and sure enough, you're right. (define) is not
>required to evaluate its first argument. Imagine
>my surprise. However, it took a close reading of
>the Denotational Semantics to discover that. Good catch!
Why were you surprised? Since define creates bindings, what sense would it
make to evaluate the thing being defined? It doesn't have a value until
define is completed.
Also, how could this syntax:
(define (<variable> <formals>) <body>)
be supported if define were supposed to evaluate its first argument? That
looks like an invocation of the function being defined, but neither the
variable nor the formals have any value at that time.
>> >is (foo a b) with a numeric a and a numeric non-zero b required
>> >to evaluate to a different number than (foo a b)?
>>
>> No. Why would it?
>
>Because under some readings of the denotational semantics,
>(foo a b) (where only one space separates "foo" from "a")
>refers to a variable whose name is spelled f, o, o
>and (foo a b) (where two spaces separate "foo" from "a")
>refers to a variable whose name is spelled f, o, o, space.
The denotational semantics doesn't cover the lexical syntax of the
language. If you go to the "Lexical conventions" section of RnRS you'll
see that whitespace is used only to separate tokens.
Some implementations may have adopted something like Lisp's syntax for
escaping special characters in symbol notation. In Lisp you could enter:
(foo| | a b)
or
(foo\ a b)
or
(|FOO | a b)
>> ??? Because it is not in R5RS, it ought to be legal?
>> That doesn't compute.
>
>Because R5RS doesn't prohibit it, and because it will not
>change the semantics of any R5RS-legal programs, I argue
>that it ought to be legal. IOW, if your implementation
>allows additional namespace in the symbols, you have the
>right to use it as you please.
I think you two were using "legal" from different points of view. It's
legal for an implementation to provide such an extension. A program that
makes use of the extension is not legal according to RnRS; it will work in
that particular implementation, but it wouldn't be portable.
--
Barry Margolin, bar...@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Bill Richter
>>>>> Dillinger <be...@sonic.net>:
>> I went back to R5RS, and sure enough, you're right. (define) is
>> not required to evaluate its first argument. Imagine my
>> surprise. However, it took a close reading of the Denotational
>> Semantics to discover that. Good catch!
Barry> Why were you surprised? Since define creates bindings, what
Barry> sense would it make to evaluate the thing being defined? It
Barry> doesn't have a value until define is completed.
Ray, I'm always thrilled when folks here read the R5RS DS, but you
don't need it. From the R5RS informal semantics:
5.2.1 Top level definitions
At the top level of a program, a definition
(define <variable> <expression>)
has essentially the same effect as the assignment expression
(set! <variable> <expression>)
if <variable> is bound. If <variable> is not bound, however, then
the definition will bind <variable> to a new location before
performing the assignment, whereas it would be an error to perform
a set! on an unbound variable.
Ray, doesn't that explain exactly what Barry said?
The R5RS DS isn't going to help that much anyway, since `define' isn't
defined, except in this set! sort of fashion, in giving the meaning of
a whole program.
Ray Dillinger
> What you wish can be done:
>
> http://groups.google.com/groups?selm=7eb8ac3e.0111051503.d8cf750%40posting.google.com
>
> The end of that article shows how to use identifiers with embedded
> spaces, and even how to use an empty string and "numbers" as
> identifiers.
Man.... that's just evil. If there were an obfuscated
scheme contest, this would definitely be in there
somewhere.
But -- and this is the important point to me -- the lexer
was working with legitimate scheme tokens that uses spaces
as delimiters only. It still doesn't have to worry about
being extended by program definitions at read time. Thank
God.
Macrology, however, is a topic of study unto itself, and
starts right after the lexer is finished.
Stephan H.M.J. Houben
>>
>> (eval `(define ,(string->symbol "foo") (lambda (a b) (+ a b)))
>> (interaction-environment))
>
>you know, I've done that trick a dozen times and never
>discovered a place where it was illegal.
What excuse for a Scheme implementation are you using?
Cast the evil thing far from you!
>So I went back
>to R5RS, and sure enough, you're right. (define) is not
>required to evaluate its first argument.
"Not allowed" would be my way to put it.
>Imagine
>my surprise. However, it took a close reading of
>the Denotational Semantics to discover that. Good catch!
>
>However, your rewrite completely misses the point of my
>question, which was about defining a variable whose name
>contains a space or a paren. A version of your rewrite
>that captures the case I'm asking about would be something
>like
>
>(eval '(define ,(string->symbol "foo ") (lambda (a b) (+ a b)))
> (interaction-environment)
>
>
>> Note that this is not required to work in an R5RS Scheme;
>Duly noted.
>> ...gotcha about case....
>Duly noted.
Moreover, you should change the quote ' into a backquote ` and
add the missing ), and then it should work.
It defines a procedure named "foo ", which cannot be directly
accessed, but you can do something like:
(eval (string->symbol "foo ") (interaction-environment))
to get at it.
And then there are some Schemes with non-standard extensions which allow
something like |foo |.
>
>> >(define foo (lambda (a b)(- a b)))
>>
>> This is what the procedure eval actually sees.
>
>No, it sees (define ##### (lambda (a b) (- a b)))
>where ##### is the symbol created by string->symbol,
>whose name is spelled f, o, o, space.
Sorry, I missed the space.
>> >is (foo a b) with a numeric a and a numeric non-zero b required
>> >to evaluate to a different number than (foo a b)?
>>
>> No. Why would it?
>
>Because under some readings of the denotational semantics,
>(foo a b) (where only one space separates "foo" from "a")
>refers to a variable whose name is spelled f, o, o
>and (foo a b) (where two spaces separate "foo" from "a")
>refers to a variable whose name is spelled f, o, o, space.
But whitespace is already interpreted by the parsing phase, and
doesn't influence the semantics of a Scheme program in any way.
(Well, except in literal strings and such, of course.)
>> >It seems to me that R5RS requires that
>> >(foo a b) evaluate differently from (((string->symbol "foo ")) a b)
>>
>> Sure, the second is an error. You try to apply a symbol. BTW, I think you
>> meant one pair of parentheses less around (string->symbol "foo"), but
>> that doesn't change the situation in this case.
>
>Uh, yes it does. Extra parens trigger an evaluation, and if
>it's a variable name, gets you the value.
I meant that it gets you the same error, since in both cases
you are trying to evaluate a symbol. Note that a variable name is something
different from a symbol; the latter is a first-class object, the first is
not.
>> >Would it be legal for the symbol created by string->symbol to use
>> >escape codes followed by numbers for characters that might cause
>> >the parser trouble otherwise (notably spaces and parens)?
>>
>> Sure, any string is legal input to string->symbol. Not every symbol
>> that can be generated by string->symbol can appear as a literal
>> in the program text.
>>
>> >For
>> >example if
>> >
>> >(eqv? (string->symbol "foo ") foo\20 ) ==> #t
>> >
>> >does it violate the semantics of R5RS?
>>
>> Yes, since \ may not appear in identifiers.
>
>R5RS does not require that an implementation allow \ to appear
>in identifiers, but it doesn't disallow it either.
So any program that uses it is not portable, since it will not
run on any R5RS implementation I throw at it.
>> >Note that allowing symbols to include a backslash would be a
>> >language extension not described in R5RS, so using backslash as
>> >an escape character would effectively create a separate namespace
>> >for string->symbol on args with troublesome characters, guaranteed
>> >not to collide with strictly R5RS symbols. So, I argue that this
>> >does not pre-empt a language construct defined by the standard and
>> >ought to be legal.
>
>> ??? Because it is not in R5RS, it ought to be legal?
>> That doesn't compute.
>
>Because R5RS doesn't prohibit it, and because it will not
>change the semantics of any R5RS-legal programs, I argue
>that it ought to be legal. IOW, if your implementation
>allows additional namespace in the symbols, you have the
>right to use it as you please.
...and make your program nonportable as a result.
>> >I hope it's legal anyway; it would make writing parsers a lot
>> >easier.
>>
>> It is an extension of some Schemes. Why would it make writing parsers
>> easier?
>
>because then the parser wouldn't have to be redefinable at
>runtime to look out for spaces or parens that, because of
>strange definitions in play, turn out not to be delimiters.
I still don't really understand this. The Scheme parser parses Scheme.
It uses the lexical rules of Scheme. If you want to parse something else,
you shouldn't be using the Scheme parser.
Stephan
Ray Dillinger
> I still don't really understand this. The Scheme parser parses Scheme.
> It uses the lexical rules of Scheme. If you want to parse something else,
> you shouldn't be using the Scheme parser.
The question, really, is about the extent to which dynamic
definitions can change the language recognized by the lexer.
The parser (which sorts tokens into structures according to
linguistic rules) starts out parsing scheme, but as it takes
various macro definitions, the language it parses changes.
The lexer, on the other hand (which just reads & identifies
tokens), looks like it always lexes the same language.
And that was the crucial point I was asking about -- whether
I need to make any allowance in a scheme lexer for weird
situations in which whitespace is anything other than part
of a string, comment, or intertoken delimiter. Or whether a
scheme lexer ever needs to make any allowance for a paren
outside of a comment or a string which is not a structural
paren. Or etc.
Bear