Please, note that I am entirely open for every points on this proposal (which I do not dare yet to call PEP).
Abstract ========
This proposal suggests to add symbols into Python.
Symbols are objects whose representation within the code is more important than their actual value. Two symbols needs only to be equally-comparable. Also, symbols need to be hashable to use as keys of dictionary (symbols are immutable objects).
Motivations ===========
Currently, there is no obvious way to define constants or states or whatever would be best represented by symbols. Discussions on comp.lang.python shows at least half a dozen way to replace symbols.
Some use cases for symbols are : state of an object (i.e. for a file opened/closed/error) and unique objects (i.e. attributes names could be represented as symbols).
Many languages propose symbols or obvious ways to define symbol-like values. For examples in common languages:
In C/C++ : Symbols are emulated using Enums. In Haskell/OCaml : Symbols are defined by union types with empty constructors. Symbols are local to modules. In Prolog : Symbols are called atoms ... they are local to modules (at least in swi-prolog) In Ruby : Symbols are introduced be the ":" notation. ":open" is a symbol. Symbols are global. In LISP : Symbols are introduced by "'". "'open" is a symbol. Symbols are local to modules.
Proposal ========
First, I think it would be best to have a syntax to represent symbols. Adding some special char before the name is probably a good way to achieve that : $open, $close, ... are $ymbols.
On the range of symbols, I think they should be local to name space (this point should be discussed as I see advantages and drawbacks for both local and global symbols). For example, for the state of the file object I would write :
>>> file.$open, file.$close, file.$error
Then, given some other objects (say some other device which also may be opened) :
>>> assert f.state != dev.state
would always hold if both objects use locally-defined symbols. The only way for these states to be equal would be, for example, for the device object to explicitly assign the file symbols :
>>> dev.state = file.$open
By default, symbols should be local to the current module. Then, being in the module "device_manager", this would hold:
>>> assert $opened == device_manager.$opened
There should be a way to go from strings to symbols and the other way around. For that purpose, I propose:
One possible way to implement symbols is simply with integers resolved as much as possible at compile time.
The End =======
Thanks to those who read entirely this proposal and I hope this proposal will gather enough interests to become a PEP and someday be implemented, maybe (probably?) in a completely different way ;)
Pierre Barbier de Reuille <pierre.barb...@cirad.fr> wrote:
> This proposal suggests to add symbols into Python.
I still don't think "symbol" is particularly descriptive as a name; there are too many other things already in the language that might also be called a "symbol".
> Symbols are objects whose representation within the code is more > important than their actual value.
An interesting
> Two symbols needs only to be equally-comparable.
I believe it would be more useful to have enumerated types in Python, which would also allow values from the same type to be cmp() compared.
> Currently, there is no obvious way to define constants or states or > whatever would be best represented by symbols.
"constants" isn't a good equivalent here, since constants in most other languages are all about the name-to-value mapping, which you said was unimportant for this concept.
> Discussions on comp.lang.python shows at least half a dozen way to > replace symbols.
s/replace/implement/
> Some use cases for symbols are : state of an object (i.e. for a file > opened/closed/error) and unique objects (i.e. attributes names could > be represented as symbols).
That pretty much covers the common use cases. Nicely done.
> First, I think it would be best to have a syntax to represent > symbols.
I disagree. Namespaces would be fine, and would also make clear which values were related to each other; e.g. for your "state of an object" use case, it's useful to have all the states in one namespace, separate from unrelated states of other classes of objects.
> Adding some special char before the name is probably a good way to > achieve that : $open, $close, ... are $ymbols.
thefile.state = FileState.open if thefile.state == FileState.closed: print "File is closed"
So all that's needed here is the type SomeTypeDefiningStates, not a new syntax.
> One possible way to implement symbols is simply with integers > resolved as much as possible at compile time.
I believe all your requirements and motivations could be met with an Enum type in the language. Here's an implementation using a sequence of integers for the underlying values:
An enumerated type would also allow values from that type to be compared with cmp() if their sequence was considered important. e.g. for object state, the "normal" sequence of states could be represented in the enumeration, and individual states compared to see if they are "later" that each other. If sequence was not considered important, of course, this feature would not get in the way.
-- \ "I put instant coffee in a microwave oven and almost went back | `\ in time." -- Steven Wright | _o__) | Ben Finney
Pierre Barbier de Reuille <pierre.barb...@cirad.fr> writes:
> Please, note that I am entirely open for every points on this proposal > (which I do not dare yet to call PEP).
> Abstract > ========
> This proposal suggests to add symbols into Python.
You're also proposing adding a syntax to generate symbols. If so, it's an important distinction, as simply addig symbols is a lot more straightforward than adding new syntax.
> Symbols are objects whose representation within the code is more > important than their actual value. Two symbols needs only to be > equally-comparable. Also, symbols need to be hashable to use as keys of > dictionary (symbols are immutable objects).
The values returned by object() meet this criteria. You could write LISPs gensym as:
gensym = object
As you've indicated, there are a number of ways to get such objects. If all you want is symbols, all that really needs to happen is that one of those ways be blessed by including an implementation in the distribution.
> In LISP : Symbols are introduced by "'". "'open" is a symbol.
No, they're not. "'(a b c)" is *not* a symbol, it's a list. Symbols in LISP are just names. "open" is a symbol, but it's normally evaluated. The "'" is syntax that keeps the next expression from being evaluated, so that "'open" gets you the symbol rather than it's value. Since you're trying to introduce syntax, I think it's important to get existing practice in other languages right.
> Proposal > ========
> First, I think it would be best to have a syntax to represent symbols.
That's half the proposal.
> Adding some special char before the name is probably a good way to > achieve that : $open, $close, ... are $ymbols.
$ has bad associations for me - and for others that came from an earlier P-language. Also, I feel that using a magic character to introduce type information doesn't feel very Pythonic.
While you don't make it clear, it seems obvious that you intend that if $open occurs twice in the same scope, it should refer to the same symbol. So you're using the syntax for a dual purpose. $name checks to see if the symbol name exists, and references that if so. If not, it creates a new symbol and with that name. Having something that looks like a variables that instantiates upon reference instead of raising an exception seems like a bad idea.
> On the range of symbols, I think they should be local to name space > (this point should be discussed as I see advantages and drawbacks for > both local and global symbols).
Agreed. Having one type that has different scoping rules than everything else is definitely a bad idea.
> There should be a way to go from strings to symbols and the other way > around. For that purpose, I propose:
So the heart of your proposal seems to be twofold: The addition of "symbol" as a type, and the syntax that has the lookup/create behavior I described above.
> Implementation > ==============
> One possible way to implement symbols is simply with integers resolved > as much as possible at compile time.
What exactly are you proposing be "resolved" at compile time? How is this better than using object, as illustratd above?
Suggested changes:
Provide a solid definition for the proposed builtin type "symbol". Something like:
symbol objects support two operations: is and equality comparison. Two symbol objects compare equal if and only if they are the same object, and symbol objects never compare equal to any other type of object. The result of other operations on a symbol object is undefined, and should raise a TypeError exception.
symbol([value]) - creates a symbol object. Two distinct calls to symbol will return two different symbol objects unless the values passed to them as arguments are equal, in which case they return the same symbol object. If symbol is called without an argument, it returns a unique symbol.
I left the type of the value argument unspecified on purpose. Strings are the obvious type, but I think it should be as unrestricted as possible. The test on value is equality, not identity, because two strings can be equal without being the same string, and we want that case to give us the same symbol. I also added gensym-like behavior, because it seemed useful. You could do without equality comparison, but it seems like a nice thing to have.
Now propose a new syntax that "means" symbol, ala {} "meaning" dict and [] "meaning" list. Don't use "$name" (& and ^ are also probably bad, but not as; pretty much everything else but ? is already in use). Python does seem to be moving away from this kind of thing, though.
Personally, I think that the LISP quote mechanism would be a better addition as a new syntax, as it would handle needs that have caused a number of different proposals to be raised. It would require that symbol know about the internals of the implementation so that ?name and symbol("name") return the same object, and possibly exposing said object to the programmer. And this is why the distinction about how LISP acts is important.
<mike -- Mike Meyer <m...@mired.org> http://www.mired.org/home/mwm/ Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
On Sat, 12 Nov 2005 16:52:12 -0500, Mike Meyer <m...@mired.org> wrote:
[...]
>Personally, I think that the LISP quote mechanism would be a better >addition as a new syntax, as it would handle needs that have caused a >number of different proposals to be raised. It would require that >symbol know about the internals of the implementation so that ?name >and symbol("name") return the same object, and possibly exposing said >object to the programmer. And this is why the distinction about how >LISP acts is important.
I wonder if the backquote could be deprecated and repurposed. It could typographically serve nicely as a lisp quote then. But in python, how would 'whatever be different from lambda:whatever ? (where of course whatever could be any expression parenthesized as necessary)
On Sat, 12 Nov 2005 18:59:39 +0100, Pierre Barbier de Reuille wrote:
> First, I think it would be best to have a syntax to represent symbols. > Adding some special char before the name is probably a good way to > achieve that : $open, $close, ... are $ymbols.
I think your chances of convincing Guido to introduce new syntax is slim to none. (Not quite zero -- he did accept @ for decorators.)
I think symbols should simply be an immutable object, one with state and limited or no behaviour, rather than a brand new syntactical element. Being an object, you can reference them in whatever namespace you define them in.
Personally, I think rather than adding a new language feature (...slim to none...) there is more hope of getting something like this added to the standard library:
> Pierre Barbier de Reuille <pierre.barb...@cirad.fr> wrote:
>>This proposal suggests to add symbols into Python.
> I still don't think "symbol" is particularly descriptive as a name; > there are too many other things already in the language that might > also be called a "symbol".
Well, that's the name in many languages. Then, probably all the things already in the language that might be called "symbol" may be implemented using the symbols in this proposal ... or maybe I don't see what you mean here ?
[...] >>First, I think it would be best to have a syntax to represent >>symbols.
> I disagree. Namespaces would be fine, and would also make clear which > values were related to each other; e.g. for your "state of an object" > use case, it's useful to have all the states in one namespace, > separate from unrelated states of other classes of objects.
>>Adding some special char before the name is probably a good way to >>achieve that : $open, $close, ... are $ymbols.
>>One possible way to implement symbols is simply with integers >>resolved as much as possible at compile time.
> I believe all your requirements and motivations could be met with an > Enum type in the language. Here's an implementation using a sequence > of integers for the underlying values:
> An enumerated type would also allow values from that type to be > compared with cmp() if their sequence was considered important. e.g. > for object state, the "normal" sequence of states could be represented > in the enumeration, and individual states compared to see if they are > "later" that each other. If sequence was not considered important, of > course, this feature would not get in the way.
Well, I don't think enumarated objects ARE symbols. I can see two "problems" : 1 - in the implementation, trying to compare values from different groups raises an error instead of simply returning "False" (easy to fix ...) 2 - You have to declare these enumerable variables, which is not pythonic IMO (impossible to fix ... needs complete redesign)
In the end, I really think symbols and enum are of different use, one of the interest un symbols being to let the compiler does what he wants (i.e. probably what is the most efficient).
> Pierre Barbier de Reuille <pierre.barb...@cirad.fr> writes:
>>Please, note that I am entirely open for every points on this proposal >>(which I do not dare yet to call PEP).
>>Abstract >>========
[...]
>>Symbols are objects whose representation within the code is more >>important than their actual value. Two symbols needs only to be >>equally-comparable. Also, symbols need to be hashable to use as keys of >>dictionary (symbols are immutable objects).
> The values returned by object() meet this criteria. You could write > LISPs gensym as:
> gensym = object
> As you've indicated, there are a number of ways to get such > objects. If all you want is symbols, all that really needs to happen > is that one of those ways be blessed by including an implementation in > the distribution.
Well, I may rewrite the proposal, but one good thing to have is the hability to go from symbol to string and the opposite (as written below) and that is not really allowed by this implementation of symbols.
>>In LISP : Symbols are introduced by "'". "'open" is a symbol.
> No, they're not. "'(a b c)" is *not* a symbol, it's a list. Symbols in > LISP are just names. "open" is a symbol, but it's normally evaluated. > The "'" is syntax that keeps the next expression from being evaluated, > so that "'open" gets you the symbol rather than it's value. Since > you're trying to introduce syntax, I think it's important to get > existing practice in other languages right.
You're right ! I was a bit quick here ... "'" is a way to stop evaluation and you may also write "(quote open)" for "'open".
>>First, I think it would be best to have a syntax to represent symbols.
> That's half the proposal.
>>Adding some special char before the name is probably a good way to >>achieve that : $open, $close, ... are $ymbols.
> $ has bad associations for me - and for others that came from an > earlier P-language. Also, I feel that using a magic character to > introduce type information doesn't feel very Pythonic.
> While you don't make it clear, it seems obvious that you intend that > if $open occurs twice in the same scope, it should refer to the same > symbol. So you're using the syntax for a dual purpose. $name checks to > see if the symbol name exists, and references that if so. If not, it > creates a new symbol and with that name. Having something that looks > like a variables that instantiates upon reference instead of raising > an exception seems like a bad idea.
Well, that's why symbols are absolutely not variables. One good model (IMO) is LISP symbols. Symbols are *values* and equality is not depending on the way you obtained the symbol :
>>On the range of symbols, I think they should be local to name space >>(this point should be discussed as I see advantages and drawbacks for >>both local and global symbols).
> Agreed. Having one type that has different scoping rules than > everything else is definitely a bad idea.
>>There should be a way to go from strings to symbols and the other way >>around. For that purpose, I propose:
> So the heart of your proposal seems to be twofold: The addition of > "symbol" as a type, and the syntax that has the lookup/create behavior > I described above.
>>One possible way to implement symbols is simply with integers resolved >>as much as possible at compile time.
> What exactly are you proposing be "resolved" at compile time? How is > this better than using object, as illustratd above?
> Suggested changes:
> Provide a solid definition for the proposed builtin type "symbol". > Something like:
> symbol objects support two operations: is and equality > comparison. Two symbol objects compare equal if and only if > they are the same object, and symbol objects never compare > equal to any other type of object. The result of other > operations on a symbol object is undefined, and should raise > a TypeError exception.
> symbol([value]) - creates a symbol object. Two distinct > calls to symbol will return two different symbol objects > unless the values passed to them as arguments are equal, in > which case they return the same symbol object. If symbol is > called without an argument, it returns a unique symbol.
Good definition to me !
> I left the type of the value argument unspecified on purpose. Strings > are the obvious type, but I think it should be as unrestricted as > possible. The test on value is equality, not identity, because two > strings can be equal without being the same string, and we want that > case to give us the same symbol. I also added gensym-like behavior, > because it seemed useful. You could do without equality comparison, > but it seems like a nice thing to have. > Now propose a new syntax that "means" symbol, ala {} "meaning" dict > and [] "meaning" list. Don't use "$name" (& and ^ are also probably > bad, but not as; pretty much everything else but ? is already in > use). Python does seem to be moving away from this kind of thing, > though.
Well, maybe we should find some other way to express symbols. The only thing I wanted was a way easy to write, avoiding the need to declare symbols, and allowing the specification of the scope of the symbol. My prefered syntax would be something like :
'opened, `opened or `opened`
However, none are usable in current Python.
> Personally, I think that the LISP quote mechanism would be a better > addition as a new syntax, as it would handle needs that have caused a > number of different proposals to be raised. It would require that > symbol know about the internals of the implementation so that ?name > and symbol("name") return the same object, and possibly exposing said > object to the programmer. And this is why the distinction about how > LISP acts is important.
> <mike
Maybe, although I may say I cannot see clearly how LISP quote mechanism translates into Python.
Pierre Barbier de Reuille <pierre.barb...@cirad.fr> writes:
>>>In LISP : Symbols are introduced by "'". "'open" is a symbol. >> No, they're not. "'(a b c)" is *not* a symbol, it's a list. Symbols in >> LISP are just names. "open" is a symbol, but it's normally evaluated. >> The "'" is syntax that keeps the next expression from being evaluated, >> so that "'open" gets you the symbol rather than it's value. Since >> you're trying to introduce syntax, I think it's important to get >> existing practice in other languages right. > You're right ! I was a bit quick here ... "'" is a way to stop > evaluation and you may also write "(quote open)" for "'open".
Yup. Also notice that if you eval the symbol, you get any value that happens to be bound to it. This is irrelevant for your purposes. But the properties you're looking for are - in LISP, anyway - implementation details of how it handles names.
>> While you don't make it clear, it seems obvious that you intend that >> if $open occurs twice in the same scope, it should refer to the same >> symbol. So you're using the syntax for a dual purpose. $name checks to >> see if the symbol name exists, and references that if so. If not, it >> creates a new symbol and with that name. Having something that looks >> like a variables that instantiates upon reference instead of raising >> an exception seems like a bad idea.
> Well, that's why symbols are absolutely not variables.
If they aren't variables, they probably shouldn't *look* like variables.
>> Provide a solid definition for the proposed builtin type "symbol". >> Something like:
>> symbol objects support two operations: is and equality >> comparison. Two symbol objects compare equal if and only if >> they are the same object, and symbol objects never compare >> equal to any other type of object. The result of other >> operations on a symbol object is undefined, and should raise >> a TypeError exception.
>> symbol([value]) - creates a symbol object. Two distinct >> calls to symbol will return two different symbol objects >> unless the values passed to them as arguments are equal, in >> which case they return the same symbol object. If symbol is >> called without an argument, it returns a unique symbol.
> Good definition to me !
Note that this definition doesn't capture the name-space semantics you asked for - symbol(value) is defined to return the same symbol everywhere it's called, so long as value is equal. This is probably a good thing. Using the ability to have non-strings for value means you can get this behavior by passing in something that's unique to the namespace as part of value. Said something probably depends on the the flavor of the namespace in question. This allows you to tailor the namespace choice to your needs.
Also, since I'm allowing non-strings for value, just invoking str on the symbol isn't really sufficient. Let's add an attribute 'value', such that symbol(stuff).value is identical to stuff. I you want, define symbol.__str__ as str(symbol.value) so that str(symbol("foo")) returns "foo".
> Well, maybe we should find some other way to express symbols. The only > thing I wanted was a way easy to write, avoiding the need to declare > symbols, and allowing the specification of the scope of the symbol. My > prefered syntax would be something like : > 'opened, `opened or `opened` > However, none are usable in current Python.
Well, symbol('opened') solves the declaration issue, but it's not as easy as you'd like.
>> Personally, I think that the LISP quote mechanism would be a better >> addition as a new syntax, as it would handle needs that have caused a >> number of different proposals to be raised. It would require that >> symbol know about the internals of the implementation so that ?name >> and symbol("name") return the same object, and possibly exposing said >> object to the programmer. And this is why the distinction about how >> LISP acts is important. > Maybe, although I may say I cannot see clearly how LISP quote mechanism > translates into Python.
It compiles the quoted expression and returns a code object. I'd love to recycle backquotes so that `expr` means compile(expr, 'quoted-expr', 'eval'), but that won't happen anytime soon.
Hmm. You know, $symbol$ doesn't seem nearly as bad as $symbol. It tickles TeX, not P***. I could live with that.
Like I said, the tricky part of doing this is getting `symbol` to have the semantics you want. If you compile the same string twice, you get two different code objects, though they compare equal, and the variable names in co_names are the same strings. Maybe equality is sufficient, and you don't need identity.
<mike -- Mike Meyer <m...@mired.org> http://www.mired.org/home/mwm/ Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
> Pierre Barbier de Reuille <pierre.barb...@cirad.fr> writes:
>>>While you don't make it clear, it seems obvious that you intend that >>>if $open occurs twice in the same scope, it should refer to the same >>>symbol. So you're using the syntax for a dual purpose. $name checks to >>>see if the symbol name exists, and references that if so. If not, it >>>creates a new symbol and with that name. Having something that looks >>>like a variables that instantiates upon reference instead of raising >>>an exception seems like a bad idea.
>>Well, that's why symbols are absolutely not variables.
> If they aren't variables, they probably shouldn't *look* like > variables.
Yes, that's why we should find some way to express that.
>>>Provide a solid definition for the proposed builtin type "symbol". >>>Something like:
>>> symbol objects support two operations: is and equality >>> comparison. Two symbol objects compare equal if and only if >>> they are the same object, and symbol objects never compare >>> equal to any other type of object. The result of other >>> operations on a symbol object is undefined, and should raise >>> a TypeError exception.
>>> symbol([value]) - creates a symbol object. Two distinct >>> calls to symbol will return two different symbol objects >>> unless the values passed to them as arguments are equal, in >>> which case they return the same symbol object. If symbol is >>> called without an argument, it returns a unique symbol.
>>Good definition to me !
> Note that this definition doesn't capture the name-space semantics you > asked for - symbol(value) is defined to return the same symbol > everywhere it's called, so long as value is equal. This is probably a > good thing. Using the ability to have non-strings for value means you > can get this behavior by passing in something that's unique to the > namespace as part of value. Said something probably depends on the the > flavor of the namespace in question. This allows you to tailor the > namespace choice to your needs.
Very interesting ... that way we could get global AND local symbols ... I like it !
> Also, since I'm allowing non-strings for value, just invoking str on > the symbol isn't really sufficient. Let's add an attribute 'value', > such that symbol(stuff).value is identical to stuff. I you want, > define symbol.__str__ as str(symbol.value) so that str(symbol("foo")) > returns "foo".
>>Well, maybe we should find some other way to express symbols. The only >>thing I wanted was a way easy to write, avoiding the need to declare >>symbols, and allowing the specification of the scope of the symbol. My >>prefered syntax would be something like : >>'opened, `opened or `opened` >>However, none are usable in current Python.
> Well, symbol('opened') solves the declaration issue, but it's not as > easy as you'd like.
>>>Personally, I think that the LISP quote mechanism would be a better >>>addition as a new syntax, as it would handle needs that have caused a >>>number of different proposals to be raised. It would require that >>>symbol know about the internals of the implementation so that ?name >>>and symbol("name") return the same object, and possibly exposing said >>>object to the programmer. And this is why the distinction about how >>>LISP acts is important.
>>Maybe, although I may say I cannot see clearly how LISP quote mechanism >>translates into Python.
> It compiles the quoted expression and returns a code object. I'd love > to recycle backquotes so that `expr` means > compile(expr, 'quoted-expr', 'eval'), but that won't happen anytime soon.
> Hmm. You know, $symbol$ doesn't seem nearly as bad as $symbol. It > tickles TeX, not P***. I could live with that.
Yep, I like this $symbol$ notation ! It could me equivalent to :
symbol( "symbol" )
And $object.symbol$ could translate into :
symbol( (object, "symbol") )
> Like I said, the tricky part of doing this is getting `symbol` to have > the semantics you want. If you compile the same string twice, you get > two different code objects, though they compare equal, and the > variable names in co_names are the same strings. Maybe equality is > sufficient, and you don't need identity.
> <mike
Yep, that's something I always found strange but I think this is for optimization reasons. However, with symbols the problem is quite different and we can take some time to ensure there are never two same objects with different ids ... then, we can also garanty only the use of "==" and not of "is" ...
On Sun, 13 Nov 2005 10:11:04 +0100, Pierre Barbier de Reuille wrote:
> The problem, IMHO, is that way you need to declare "symbols" > beforehands, that's what I was trying to avoid by requiring a new syntax.
???
If you don't declare your symbols, how will you get the ones that you want?
I don't understand why it is a problem to declare them first, and if it is a problem, what your solution would be.
[snip]
> Well, I don't think enumarated objects ARE symbols. I can see two > "problems" : > 1 - in the implementation, trying to compare values from different > groups raises an error instead of simply returning "False" (easy to fix ...)
As you say, that's easy to fix.
> 2 - You have to declare these enumerable variables, which is not > pythonic IMO (impossible to fix ... needs complete redesign)
Are you suggesting that the Python language designers should somehow predict every possible symbol that anyone in the world might ever need, and build them into the language as predefined things?
If that is not what you mean, can you explain please, because I'm confused.
> On Sun, 13 Nov 2005 10:11:04 +0100, Pierre Barbier de Reuille wrote:
>>The problem, IMHO, is that way you need to declare "symbols" >>beforehands, that's what I was trying to avoid by requiring a new syntax.
> ???
> If you don't declare your symbols, how will you get the ones that you want?
> I don't understand why it is a problem to declare them first, and if it is > a problem, what your solution would be.
Well, just as Python do not need variable declaration, you can just *use* them ... in dynamic languages using symbols, they just get created when used (i.e. have a look at LISP or Ruby).
>>Well, I don't think enumarated objects ARE symbols. I can see two >>"problems" : >> 1 - in the implementation, trying to compare values from different >>groups raises an error instead of simply returning "False" (easy to fix ...)
> As you say, that's easy to fix.
>> 2 - You have to declare these enumerable variables, which is not >>pythonic IMO (impossible to fix ... needs complete redesign)
> Are you suggesting that the Python language designers should somehow > predict every possible symbol that anyone in the world might ever need, > and build them into the language as predefined things?
> If that is not what you mean, can you explain please, because I'm confused.
Well, the best I can propose is for you to read the discussion with Mike Meyer. He pointer out the flaws in my proposal and we're trying to precise things.
Steven D'Aprano <st...@removethiscyber.com.au> wrote: > On Sun, 13 Nov 2005 10:11:04 +0100, Pierre Barbier de Reuille wrote: > > The problem, IMHO, is that way you need to declare "symbols" > > beforehands, that's what I was trying to avoid by requiring a new > > syntax.
> If you don't declare your symbols, how will you get the ones that > you want? > [...] > Are you suggesting that the Python language designers should somehow > predict every possible symbol that anyone in the world might ever > need, and build them into the language as predefined things?
I believe Pierre is looking for a syntax that will save him from assigning values to names; that Python will simply assign arbitrary unique values for these special names. My understanding of the intended use is that their only purpose is to compare differently to other objects of the same type, so the actual values don't matter.
What I still don't understand is why this justifies additional syntax baggage in the language, rather than an explicit assignment earlier in the code.
-- \ "Smoking cures weight problems. Eventually." -- Steven Wright | `\ | _o__) | Ben Finney
Pierre Barbier de Reuille <pierre.barb...@cirad.fr> wrote:
> Mike Meyer a écrit : > > Hmm. You know, $symbol$ doesn't seem nearly as bad as $symbol. It > > tickles TeX, not P***. I could live with that. > Yep, I like this $symbol$ notation !
Gets a big -1 here.
I've yet to see a convincing argument against simply assigning values to names, then using those names.
-- \ "Yesterday I parked my car in a tow-away zone. When I came back | `\ the entire area was missing." -- Steven Wright | _o__) | Ben Finney
> Steven D'Aprano a écrit : >> On Sun, 13 Nov 2005 10:11:04 +0100, Pierre Barbier de Reuille wrote:
>>>The problem, IMHO, is that way you need to declare "symbols" >>>beforehands, that's what I was trying to avoid by requiring a new syntax.
>> ???
>> If you don't declare your symbols, how will you get the ones that you want?
>> I don't understand why it is a problem to declare them first, and if it is >> a problem, what your solution would be.
> Well, just as Python do not need variable declaration, you can just > *use* them ... in dynamic languages using symbols, they just get created > when used (i.e. have a look at LISP or Ruby).
If you want to be technical, Python doesn't have variables. It has names and objects.
If I want a name x to be bound to an object 1, I have to define it (actually bind the name to the object):
x = 1
If I want a symbol $x$ (horrible syntax!!!) with a value 1, why shouldn't I define it using:
$x$ = 1
instead of expecting Python to somehow magically know that I wanted it? What if somebody else wanted the symbol $x$ to have the value 2 instead?
>>>Well, I don't think enumarated objects ARE symbols. I can see two >>>"problems" : >>> 1 - in the implementation, trying to compare values from different >>>groups raises an error instead of simply returning "False" (easy to fix ...)
>> As you say, that's easy to fix.
>>> 2 - You have to declare these enumerable variables, which is not >>>pythonic IMO (impossible to fix ... needs complete redesign)
>> Are you suggesting that the Python language designers should somehow >> predict every possible symbol that anyone in the world might ever need, >> and build them into the language as predefined things?
>> If that is not what you mean, can you explain please, because I'm confused.
> Well, the best I can propose is for you to read the discussion with Mike > Meyer. > He pointer out the flaws in my proposal and we're trying to precise things.
I've read the discussion, and I am no wiser.
You haven't explained why enums are not suitable to be used for symbols. You gave two "problems", one of which was "easy to fix", as you said yourself, and the other reason was that you don't want to define enums as symbols.
If you don't want to define something manually, that can only mean that you expect them to be predefined. Or am I misunderstanding something?
> Pierre Barbier de Reuille <pierre.barb...@cirad.fr> wrote:
>>Mike Meyer a écrit :
>>>Hmm. You know, $symbol$ doesn't seem nearly as bad as $symbol. It >>>tickles TeX, not P***. I could live with that.
>>Yep, I like this $symbol$ notation !
> Gets a big -1 here.
> I've yet to see a convincing argument against simply assigning values > to names, then using those names.
I can see three interests : 1 - ensure values are unique (i.e. a bit like using instances of object) 2 - values are meaningful (i.e. with introspection on the values you get a human-readable value, unlike with instances of object) 3 - getting an *easy* access to those two properties
Ben Finney <bignose+hates-s...@benfinney.id.au> writes: > I've yet to see a convincing argument against simply assigning values > to names, then using those names.
The problem with that is that you can't pass around the names of objects that are used for other things. Obviously they make enums unnecessary, but only people damaged by non-dynamic languages could think that's the main point. ;-)
Being able to do that precludes the need for converting going back and forth between strings and method names when you need to do things like keeping a list of function names, even when you need to be able to change what those function names point to.
Python doesn't really need to introduce a new type to do this. It's already there, as what we usually just call names. Probably this discussion would benefit from talking about names rather than symbols, as that seems to confuse some people.
So, Python already has symbols. What we need is a way to refer to these symbols explicitly. I would suggest to do it like in Lisp:
quote(spam)
Of course, this would preferably be implemented so that it doesn't just work on simple names:
quote(spam(eggs))
I syntactic sugar, like ' in Lisp, could be introduced later, but I don't think that would be strictly necessary.
-- Björn Lindström <b...@stp.lingfil.uu.se> Student of computational linguistics, Uppsala University, Sweden
On Mon, 14 Nov 2005 00:48:46 +1100, Ben Finney wrote: > Steven D'Aprano <st...@removethiscyber.com.au> wrote: >> On Sun, 13 Nov 2005 10:11:04 +0100, Pierre Barbier de Reuille wrote: >> > The problem, IMHO, is that way you need to declare "symbols" >> > beforehands, that's what I was trying to avoid by requiring a new >> > syntax.
>> If you don't declare your symbols, how will you get the ones that >> you want? >> [...] >> Are you suggesting that the Python language designers should somehow >> predict every possible symbol that anyone in the world might ever >> need, and build them into the language as predefined things?
> I believe Pierre is looking for a syntax that will save him from > assigning values to names; that Python will simply assign arbitrary > unique values for these special names. My understanding of the > intended use is that their only purpose is to compare differently to > other objects of the same type, so the actual values don't matter.
Unless I've misunderstood something, it would be easy to modify the recipe given here to do something like that:
Days = Enum('Mo', 'Tu', 'We', 'Th', 'Fr', 'Sa', 'Su') print Days.Mo, Days.Tu # etc.
> What I still don't understand is why this justifies additional syntax > baggage in the language, rather than an explicit assignment earlier in > the code.
The only advantage would be if you want to do something like this:
MO, TU, WE, TH, FR, SA, SU = Symbols()
and have it magically work. I can see the advantage of that, and you don't even need new syntax, just a magic function that somehow knows how many names are on the left hand side of the assignment.
This is a poor substitute:
MO, TU, WE, TH, FR, SA, SU = range(7)
Firstly, if you change the number of symbol names, you have to manually adjust the argument to range. Secondly, your symbols are actually ints, and so will compare the same way ints compare.
I don't know enough about the Python internals to tell: is there any feasible way for there to be a magic function like Symbol above that knew how many names were waiting for it to be supplied? If there is no way to do this from Python itself, it is possible to patch the compiler to do so?
> On Sun, 13 Nov 2005 12:33:48 +0100, Pierre Barbier de Reuille wrote:
>>Steven D'Aprano a écrit : >>[...]
> If you want to be technical, Python doesn't have variables. It has names > and objects.
> If I want a name x to be bound to an object 1, I have to define it > (actually bind the name to the object):
> x = 1
> If I want a symbol $x$ (horrible syntax!!!) with a value 1, why shouldn't > I define it using:
> $x$ = 1
> instead of expecting Python to somehow magically know that I wanted it? > What if somebody else wanted the symbol $x$ to have the value 2 instead?
Well, as stated, I don't care about the actual value of symbols. They *are* values. A trivial implementation of symbols are strings :
$x$ <=> "x"
However, that won't fit because of the scope, because it would be great to use "is" instead of "==" (even if not necessary), and as said Mike, you might want something else than a string. That's why `x` would be a good wawy to write that.
>>>[snip]
> I've read the discussion, and I am no wiser.
> You haven't explained why enums are not suitable to be used for symbols. > You gave two "problems", one of which was "easy to fix", as you said > yourself, and the other reason was that you don't want to define enums as > symbols.
> If you don't want to define something manually, that can only mean that > you expect them to be predefined. Or am I misunderstanding something?
Well, I suspect Python will know them, exactly as it know "without defining it" that "foo" is the string with chars f, o, o, that 3 is the number 3, that [1,2] is the list with 1 and 2, ... However, to get quicker, symbols could be created at compile-time when possible (like variables). The fact is, symbols allow compilation optimisations that you cannot get with regular types, because the language is completely free about their representations. Then, to the programmer it is a good way to have a meaningful value without caring about how to represent it in the computer. That way, while debugging, if I ask the value of file.state I will get something I can read instead of some meaningless integer or other anonymous object.
So I gain in readability of my code and in debugging capacity.
> Ben Finney <bignose+hates-s...@benfinney.id.au> writes:
>>I've yet to see a convincing argument against simply assigning values >>to names, then using those names.
> The problem with that is that you can't pass around the names of objects > that are used for other things. Obviously they make enums unnecessary, > but only people damaged by non-dynamic languages could think that's the > main point. ;-)
> Being able to do that precludes the need for converting going back and > forth between strings and method names when you need to do things like > keeping a list of function names, even when you need to be able to > change what those function names point to.
> Python doesn't really need to introduce a new type to do this. It's > already there, as what we usually just call names. Probably this > discussion would benefit from talking about names rather than symbols, > as that seems to confuse some people.
> So, Python already has symbols. What we need is a way to refer to these > symbols explicitly. I would suggest to do it like in Lisp:
> quote(spam)
> Of course, this would preferably be implemented so that it doesn't just > work on simple names:
> quote(spam(eggs))
> I syntactic sugar, like ' in Lisp, could be introduced later, but I > don't think that would be strictly necessary.
Well, if this already exists in Python's internals, then, it would be great just to expose them. Now, just being able to write :
>>> quote(spam)
quote(spam)
requires a new syntax so that spam is not resolved *before* calling the quote method.
> I've yet to see a convincing argument against simply assigning values > to names, then using those names.
I don't like any syntax I've seen so far, but I can understand the problem. If you have a name, you can redefine a name, therefore the value a name refers to is mutable. As a result if you have 2 symbols represented by names and values, you may have two symbols with different names but the same value. Hence the two "symbols" are no longer unique)
Conversely consider "NAME" to be a symbol. I can't modify "NAME". It always means the same as "NAME" and "NAME", but is never the same as "FRED". What's tricky is I can't have namespaceOne."NAME" [1] and namespaceTwo."NAME" as different "NAME"s even though logically there's no reason I couldn't treat "NAME" differently inside each.
[1] Supposing for a moment that I could have a string as a name in a namespace. (Rather than a string used as a key in that namespace)
However it might be useful to note that these two values (or symbols) are actually different, even if you remove their namespaces.
To me, the use of a symbol implies a desire for a constant, and then to only use that constant rather than the value. In certain situations it's the fact that constant A is not the same as constant B that's important (eg modelling state machines).
Often you can use strings for that sort of thing, but unfortunately even python's strings can't be used as symbols that are always the same thing in all ways. For example, we can force the id of identical strings to be different:
>>> s = "h"*10000 >>> x = "h"*10000 >>> id(s), id(x)
(135049832, 135059864)
As a result I can see that *IF* you really want this kind of symbol, rather than the various other kinds people have discussed in the thread, that some special syntax (like u'hello' for unicode 'hello') could be useful.
However, I'd be more interested in some real world usecases where this would be beneficial, and then seeing what sort of syntax would be nice/useful (Especially since I can think of some uses where it would be nice).
On the original syntax proposal, I'm firmly in the -1 camp - to me having done lots of perl in the past $foo looks very firmly like a mutable, rather than an immutable.
The reason I'm more interested in seeing usecases, is because I'd rather see where the existing approaches people use/define symbols has caused the OP problems to the extent he feels the language needs to change to fix these real world problems.
Steven D'Aprano <st...@removethiscyber.com.au> wrote: > On Mon, 14 Nov 2005 00:48:46 +1100, Ben Finney wrote: > > I believe Pierre is looking for a syntax that will save him from > > assigning values to names; that Python will simply assign > > arbitrary unique values for these special names.
> > What I still don't understand is why this justifies additional > > syntax baggage in the language, rather than an explicit assignment > > earlier in the code.
> The only advantage would be if you want to do something like this:
> MO, TU, WE, TH, FR, SA, SU = Symbols()
I believe that's exactly what Pierre doesn't want to do. He wants to simply use names (marked special in some way) and have Python automatically determine a unique value for each name, with nary an assignment in sight.
To me, that's a net loss. It makes names more complicated, it loses "explicit is better than implicit", and it loses the checks Python could do against using a name that hasn't been assigned a value (caused by e.g. a misspelled name).
-- \ "Why should I care about posterity? What's posterity ever done | `\ for me?" -- Groucho Marx | _o__) | Ben Finney
Ben Finney wrote: > I believe that's exactly what Pierre doesn't want to do. He wants to > simply use names (marked special in some way) and have Python > automatically determine a unique value for each name, with nary an > assignment in sight.
> To me, that's a net loss. It makes names more complicated, it loses > "explicit is better than implicit", and it loses the checks Python > could do against using a name that hasn't been assigned a value > (caused by e.g. a misspelled name).
I agree. And, when done explicitly, it's already easy enough to do this within the language, by just assigning it a value, even if it's an integer from range/xrange or a new sentinel like object().
-- Erik Max Francis && m...@alcyone.com && http://www.alcyone.com/max/ San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis The stairs of the sky are let down for him that he may ascend thereon to heaven. O gods, put your arms under the king: raise him, lift him
Michael <m...@cerenity.org> wrote: > Ben Finney wrote: > > I've yet to see a convincing argument against simply assigning > > values to names, then using those names.
> If you have a name, you can redefine a name, therefore the value a > name refers to is mutable.
Since there are mutable and immutable values, it might be clearer to say "the binding of a name to a value can be changed". Yes?
In that case, I don't see why someone who wants such a binding to be unchanging can't simply avoid changing it. Where's the case for having Python enforce this?
> Conversely consider "NAME" to be a symbol. I can't modify "NAME". It > always means the same as "NAME" and "NAME", but is never the same as > "FRED". What's tricky is I can't have namespaceOne."NAME" [1] and > namespaceTwo."NAME" as different "NAME"s even though logically > there's no reason I couldn't treat "NAME" differently inside each.
So you want to mark such objects as being in a namespace, where they compare the same within that namespace but not outside. Why is separate syntax necessary for this? A data type that is informed of its "space", and refuses comparison with values from other spaces, would suffice.
class Xenophobe(object): def __init__(self, space, value): self.__space = space self.__value = value
def __str__(self): return str(self.__value)
def __cmp__(self, other): if not isinstance(other, Xenophobe): raise AssertionError, \ "Can only compare Xenophobes to each other" if not other.__space == self.__space: raise AssertionError, \ "Can only compare Xenophobes from the same space" return cmp(self.__value, other.__value)
With the bonus that you could pass such values around between different names, and they'd *still* compare, or not, as you choose when you first create them.
Replace the AssertionError with some appropriate return value, if you want such comparisons to succeed.
> However it might be useful to note that these two values (or > symbols) are actually different, even if you remove their > namespaces.
The above Xenophobe implementation creates objects that know their "space" forever.
> To me, the use of a symbol implies a desire for a constant, and then > to only use that constant rather than the value. In certain > situations it's the fact that constant A is not the same as constant > B that's important (eg modelling state machines).
Since the actual value can't be easily accessed, the only purpose of a Xenophobe is too be created and compared to others.
> Often you can use strings for that sort of thing, but unfortunately > even python's strings can't be used as symbols that are always the > same thing in all ways. For example, we can force the id of > identical strings to be different:
Hold up -- what in your stated use case requires *identity* to be the same? You said you just wanted to compare them to each other.
Besides, Python *does* preserve identity for short strings...
> >>> s = "h"*10000 > >>> x = "h"*10000 > >>> id(s), id(x) > (135049832, 135059864)
... which is sufficient for unique values, if you're not actively fighting the system as above. If the use case is for brief, identical values, we have those: short strings.
> As a result I can see that *IF* you really want this kind of symbol, > rather than the various other kinds people have discussed in the > thread, that some special syntax (like u'hello' for unicode 'hello') > could be useful.
I can see the case for a new type. I disagree that syntax changes are necessary.
> However, I'd be more interested in some real world usecases where > this would be beneficial, and then seeing what sort of syntax would > be nice/useful (Especially since I can think of some uses where it > would be nice).
Real use cases would interest me, too, but *only* if they can't be satisfied with a new type that knows things about its creation state, such as Xenophobe.
> The reason I'm more interested in seeing usecases, is because I'd > rather see where the existing approaches people use/define symbols > has caused the OP problems to the extent he feels the language needs > to change to fix these real world problems.
Ditto.
-- \ "My aunt gave me a walkie-talkie for my birthday. She says if | `\ I'm good, she'll give me the other one next year." -- Steven | _o__) Wright | Ben Finney
>>>I've yet to see a convincing argument against simply assigning >>>values to names, then using those names.
>>If you have a name, you can redefine a name, therefore the value a >>name refers to is mutable.
> Since there are mutable and immutable values, it might be clearer to > say "the binding of a name to a value can be changed". Yes?
> In that case, I don't see why someone who wants such a binding to be > unchanging can't simply avoid changing it. Where's the case for having > Python enforce this?
The problem is not about having something constant ! The main point with symbols is to get human-readable values. Let say you have a symbol "opened" and a symbol "closed". The state of a file may be one of the two.
If you have these symbols, you can ask for the state at any point and get something readable. If you use constants valued, typically, to integers, the state of your file will we 0 or 1, which does not mean anything.
Now, if you're using an object with more than two states, and moreover if the number of states is likely to increase during developpement, it's much more convenient to directly get the *meaning* of the value rather than the value itself (which does not mean anything).
The key point that, I think, you misunderstand is that symbols are not *variables* they are *values*.
>>Conversely consider "NAME" to be a symbol. I can't modify "NAME". It >>always means the same as "NAME" and "NAME", but is never the same as >>"FRED". What's tricky is I can't have namespaceOne."NAME" [1] and >>namespaceTwo."NAME" as different "NAME"s even though logically >>there's no reason I couldn't treat "NAME" differently inside each.
> So you want to mark such objects as being in a namespace, where they > compare the same within that namespace but not outside. Why is > separate syntax necessary for this? A data type that is informed of > its "space", and refuses comparison with values from other spaces, > would suffice.
> class Xenophobe(object): > def __init__(self, space, value): > self.__space = space > self.__value = value
> def __str__(self): > return str(self.__value)
> def __cmp__(self, other): > if not isinstance(other, Xenophobe): > raise AssertionError, \ > "Can only compare Xenophobes to each other" > if not other.__space == self.__space: > raise AssertionError, \ > "Can only compare Xenophobes from the same space" > return cmp(self.__value, other.__value)
> With the bonus that you could pass such values around between > different names, and they'd *still* compare, or not, as you choose > when you first create them.
> Replace the AssertionError with some appropriate return value, if you > want such comparisons to succeed.
Well, I think a new syntax will promote the use of symbols. And as I think they are good practice (much better than meaningless constants) they should be promoted. Needless to say that in every language I know implementing symbols (or something close to symbols), there is an easy-to-use syntax associated.
>>However it might be useful to note that these two values (or >>symbols) are actually different, even if you remove their >>namespaces.
> The above Xenophobe implementation creates objects that know their > "space" forever.
>>To me, the use of a symbol implies a desire for a constant, and then >>to only use that constant rather than the value. In certain >>situations it's the fact that constant A is not the same as constant >>B that's important (eg modelling state machines).
> Since the actual value can't be easily accessed, the only purpose of a > Xenophobe is too be created and compared to others.
>>Often you can use strings for that sort of thing, but unfortunately >>even python's strings can't be used as symbols that are always the >>same thing in all ways. For example, we can force the id of >>identical strings to be different:
> Hold up -- what in your stated use case requires *identity* to be the > same? You said you just wanted to compare them to each other.
> Besides, Python *does* preserve identity for short strings...
> ... which is sufficient for unique values, if you're not actively > fighting the system as above. If the use case is for brief, identical > values, we have those: short strings.
Well, one *big* difference between short string and symbols is that the identity between short strings are implementation dependant, while between symbols it has to be in all implementations as you will rely on this identity. Then, once more, strings are just one possible implementation for symbols and I wouldn't like to tie that much symbols to strings.
>>As a result I can see that *IF* you really want this kind of symbol, >>rather than the various other kinds people have discussed in the >>thread, that some special syntax (like u'hello' for unicode 'hello') >>could be useful.
> I can see the case for a new type. I disagree that syntax changes are > necessary.
Well, syntactic sugar is all about what you want to promote or discourage ... it is never necessary, even if it can really be usefull.
>>However, I'd be more interested in some real world usecases where >>this would be beneficial, and then seeing what sort of syntax would >>be nice/useful (Especially since I can think of some uses where it >>would be nice).
> Real use cases would interest me, too, but *only* if they can't be > satisfied with a new type that knows things about its creation state, > such as Xenophobe.
Well, once more, new syntax for new objects never solve new problems, they just make them easier to write.
>>The reason I'm more interested in seeing usecases, is because I'd >>rather see where the existing approaches people use/define symbols >>has caused the OP problems to the extent he feels the language needs >>to change to fix these real world problems.
> Please, note that I am entirely open for every points on this proposal > (which I do not dare yet to call PEP).
I still don't see why you can't just use strings. The only two issues I see you might have with them are a) two identical strings might not be identical by id(), b) they aren't local in scope.
The objection a) is minor. One, all of your examples use equality for testing already, and two, short strings are interned and identical in most cases anyway (they only differ if you go to lengths to create them, or they aren't sufficiently "variable like") - at most you would have to standardize the rules.
The objection b) is a little harder to dismiss. But I'm not sure if you've completely thought what it means for a symbol to be "local to a module". What happens when you assign a variable containing a symbol to a variable in another module? For that matter, what does it mean to be "in a module". Which module is a class instance (and associated sybols) "in" if the class is defined in one module, instantiated in another, and then passed as a return value to a third? What about from ... imports? If you need a symbol "from another class" what's the mechanism of obtaining it? Can you import symbols? Since you advocate storing symbols internally as integers, I suppose you would have a program-global table to keep symbols from different modules from having the same internal representation. How do you pickle a symbol and have it go to a different Python program, which may have a massive symbol table of it's own?
It's been said before, and I'll say it again - the key to successful Python language changes is compelling use cases. Find an existing Python program or library (the stdlib is best) which would be markedly improved by your language change. Not only will Guido be more likely to be convinced, but what you're proposing will likely be clearer to everyone else, if it's grounded in practical use cases.
