[Python-ideas] Support other dict types for type.__dict__

[Victor Stinner]

Hi,

I'm trying to create read-only objects using a "frozendict" class.
frozendict is a read-only dict. I would like to use frozendict for the
class dict using a metaclass, but type.__new__() expects a dict and
creates a copy of the input dict.

I would be nice to support custom dict type: OrderedDict and
frozendict for example. It looks possible to patch CPython to
implement this feature, but first I would like first to know your
opinion about this idea :-)

Victor
_______________________________________________
Python-ideas mailing list
Python...@python.org
http://mail.python.org/mailman/listinfo/python-ideas

[Chris Rebert]

On Thu, Feb 23, 2012 at 3:34 PM, Victor Stinner
<victor....@haypocalc.com> wrote:
> Hi,
>
> I'm trying to create read-only objects using a "frozendict" class.
> frozendict is a read-only dict. I would like to use frozendict for the
> class dict using a metaclass, but type.__new__() expects a dict and
> creates a copy of the input dict.

And you can't use __slots__ because...?

Cheers,
Chris

[Victor Stinner]

> And you can't use __slots__ because...?

Hum, here is an example:
---
def Enum(**kw):
class _Enum(object):
__slots__ = list(kw.keys())
def __new__(cls, **kw):
inst = object.__new__(cls)
for key, value in kw.items():
setattr(inst, key, value)
return inst
return _Enum(**kw)

components = Enum(red=0, green=1, blue=2)
print(components.red)
components.red=2
print(components.red)
components.unknown=10
---

components.unknown=10 raises an error, but not components.red=2.
__slots__ denies to add new attributes, but not to modify existing
attributes.

The idea of using a frozendict is to deny the modification of an
attribute value after the creation of the object. I don't see how to
use __slots__ to implement such constraints.

Victor

[Chris Rebert]

On Thu, Feb 23, 2012 at 4:27 PM, Victor Stinner
<victor....@haypocalc.com> wrote:
>> And you can't use __slots__ because...?

<snip>

> components.unknown=10 raises an error, but not components.red=2.
> __slots__ denies to add new attributes, but not to modify existing
> attributes.
>
> The idea of using a frozendict is to deny the modification of an
> attribute value after the creation of the object. I don't see how to
> use __slots__ to implement such constraints.

Right, stupid question; didn't think that one all the way through.

- Chris

[Nick Coghlan]

On Fri, Feb 24, 2012 at 9:34 AM, Victor Stinner
<victor....@haypocalc.com> wrote:
> Hi,
>
> I'm trying to create read-only objects using a "frozendict" class.
> frozendict is a read-only dict. I would like to use frozendict for the
> class dict using a metaclass, but type.__new__() expects a dict and
> creates a copy of the input dict.

Do you have a particular reason for doing it that way rather than just
overriding __setattr__ and __delattr__ to raise TypeError?

Or overriding the __dict__ descriptor to return a read-only proxy?

There are a *lot* of direct calls to the PyDict APIs in the object
machinery. Without benchmark results clearly showing a negligible
speed impact, I'd be -1 on increasing the complexity of all that code
(and making it slower) to support niche use cases that can already be
handled a couple of other ways.

Cheers,
Nick.

--
Nick Coghlan   |   ncog...@gmail.com   |   Brisbane, Australia

[David Townshend]

Can't this also be done using metaclasses?

[Ronny Pfannschmidt]

On 02/24/2012 01:27 AM, Victor Stinner wrote:
>> And you can't use __slots__ because...?
>
> Hum, here is an example:
> ---

note untested, since written in mail client:

class Enum(object):
__slots__ = ("_data",)
_data = WriteOnceDescr('_data') # left as exercise

def __init__(self, **kw):
self._data = frozendict(kw)

def __getattr__(self, key):
try:
return self._data[key]
except KeyError:
raise AttributeError(key)

[Simon Sapin]

Le 24/02/2012 06:33, David Townshend a écrit :
> Can't this also be done using metaclasses?

Hi,

Are you thinking of __prepare__? I did to but I read the details of this:

http://docs.python.org/py3k/reference/datamodel.html#customizing-class-creation

The class body can be executed "in" any mapping. Then I’m not sure but
it looks like type.__new__ only takes a real dict. You have to do
something in your overridden __new__ to eg. keep the OrderedDict’s order.

Regards,
--
Simon Sapin

[anatoly techtonik]

On Fri, Feb 24, 2012 at 4:08 AM, Nick Coghlan <ncog...@gmail.com> wrote:
> On Fri, Feb 24, 2012 at 9:34 AM, Victor Stinner
> <victor....@haypocalc.com> wrote:
>> Hi,
>>
>> I'm trying to create read-only objects using a "frozendict" class.
>> frozendict is a read-only dict. I would like to use frozendict for the
>> class dict using a metaclass, but type.__new__() expects a dict and
>> creates a copy of the input dict.
>
> Do you have a particular reason for doing it that way rather than just
> overriding __setattr__ and __delattr__ to raise TypeError?
>
> Or overriding the __dict__ descriptor to return a read-only proxy?
>
> There are a *lot* of direct calls to the PyDict APIs in the object
> machinery. Without benchmark results clearly showing a negligible
> speed impact, I'd be -1 on increasing the complexity of all that code
> (and making it slower) to support niche use cases that can already be
> handled a couple of other ways.

I also think about reverse process of removing things that were proved
to be underused. That probably requires AST spider that crawls
existing Python project to see how various constructs are used.
--
anatoly t.

[Victor Stinner]

>> Can't this also be done using metaclasses?

Yes, my current proof-of-concept (PoC) uses a metadata with __prepare__.

> The class body can be executed "in" any mapping. Then I’m not sure but it
> looks like type.__new__ only takes a real dict. You have to do something in
> your overridden __new__ to eg. keep the OrderedDict’s order.

type.__new__ accepts any class inheriting from dict. My frozendict PoC
inherits from dict, so it just works. But the point is that
type.__new__ makes a copy of the dict and later it is no more possible
to replace the dict.

I would like to be able to choose the type of the __dict__ of my class.

Victor

[Victor Stinner]

Hum, after thinking twice, using a "frozendict" for type.__dict__ is
maybe overkill for my needs (and intrused as noticed Nick). Attached
patch for Python 3.3 is a simpler approach: add __final__ special
value for class. If this variable is present, the type is constant.
Example:
---
class Test:
__final__=True
x = 1

Test.x = 2 # raise a TypeError
Test.new_attr = 1 # raise a TypeError
del Test.x # raise a TypeError
---

There are various ways to deny the modification of a class attribute,
but I don't know how to block the removal of an attribute of the
addition of a new attribute without my patch.

--

My patch is just a proof-of-concept. For example, it doesn't ensure
that values are read-only too. By the way, how can I check that "a
value is constant"? Except builtin immutable types, I suppose that the
only way is to call hash(obj) and excepts an expect a TypeError.

Victor

[Yury Selivanov]

On 2012-02-24, at 9:29 PM, Victor Stinner wrote:

> Hum, after thinking twice, using a "frozendict" for type.__dict__ is
> maybe overkill for my needs (and intrused as noticed Nick). Attached
> patch for Python 3.3 is a simpler approach: add __final__ special
> value for class. If this variable is present, the type is constant.
> Example:
> ---
> class Test:
> __final__=True
> x = 1

-1 on this. The next move would be adding friend classes and protected methods ;)
__setattr__ works perfectly for those purposes. Moreover, you can emulate
your idea on unpatched python by using metaclasses.

-
Yury

[Ned Batchelder]

On 2/24/2012 9:29 PM, Victor Stinner wrote:
> Hum, after thinking twice, using a "frozendict" for type.__dict__ is
> maybe overkill for my needs (and intrused as noticed Nick). Attached
> patch for Python 3.3 is a simpler approach: add __final__ special
> value for class. If this variable is present, the type is constant.

The Python answer for people who want read-only data structures has
always been, "Don't modify them if you don't want to, and write docs
that tell other people not to as well." What are you building that this
answer isn't good enough?

--Ned.

[Steven D'Aprano]

Ned Batchelder wrote:

> The Python answer for people who want read-only data structures has
> always been, "Don't modify them if you don't want to, and write docs
> that tell other people not to as well." What are you building that this
> answer isn't good enough?

That is silly. That alleged "Python answer" is like telling people that they
don't need test frameworks or debuggers because the "Python answer" for people
wanting to debug their code is not to write buggy code in the first place.

Python has read-only data structures: tuple, frozenset, str, etc. If you ask
yourself why Python has immutable types, it might give you a clue why Victor
wants the ability to create other immutable types like frozendict, and why
"don't modify them" is not a good enough answer:

- Immutable types can be used as keys in dicts.

- Immutable types protect you from errors. While you might intend not
to modify a data structure, bugs do happen.

Immutability gives you an immediate exception at the exact time and place you
attempt to modify the data structure instead of at some arbitrary time later
far from the actual bug.

Python has excellent support for read-only data structures, so long as you
write them in C.

--
Steven

[Masklinn]

On 2012-02-26, at 00:05 , Steven D'Aprano wrote:
> - Immutable types can be used as keys in dicts.

*technically*, you can use mutable types as dict keys if you define
their __hash__ no? That is of course a bad idea when the instances
are *expected* to be modified, but it should "work".

> - Immutable types protect you from errors. While you might intend not
> to modify a data structure, bugs do happen.

Immutables are also inherently thread-safe (since thread safety is about
shared state, and shared immutables are not state). Which is a nice
guarantee.

> Python has excellent support for read-only data structures, so long as you write them in C.

There's also good support of the "consenting adults" variety (use
_-prefixed attributes for the actual state and expose what needs to be
exposed via properties and methods). That can be simplified with a
custom descriptor type which can only be set once (similar to java's
`final`), it would be set in the type's constructor and never re-set
from this.

[David Townshend]

On Feb 26, 2012 1:35 AM, "Masklinn" <mask...@masklinn.net> wrote:
>
> On 2012-02-26, at 00:05 , Steven D'Aprano wrote:
> > - Immutable types can be used as keys in dicts.
>
> *technically*, you can use mutable types as dict keys if you define
> their __hash__ no? That is of course a bad idea when the instances
> are *expected* to be modified, but it should "work".

I wouldn't say this is necessarily a bad thing at all. It just depends what defines the object. If an instance represent a specific object (e.g. a database record) you wouldn't expect the hash to change if you modified an attribute of it, since the instance still represents the same object.

>
> > - Immutable types protect you from errors. While you might intend not
> >  to modify a data structure, bugs do happen.
>
> Immutables are also inherently thread-safe (since thread safety is about
> shared state, and shared immutables are not state). Which is a nice
> guarantee.
>
> > Python has excellent support for read-only data structures, so long as you write them in C.
>
> There's also good support of the "consenting adults" variety (use
> _-prefixed attributes for the actual state and expose what needs to be
> exposed via properties and methods). That can be simplified with a
> custom descriptor type which can only be set once (similar to java's
> `final`), it would be set in the type's constructor and never re-set
> from this.
>
> _______________________________________________
> Python-ideas mailing list
> Python...@python.org
> http://mail.python.org/mailman/listinfo/python-ideas

Maybe I'm missing something here, but what's wrong with just using __getattr__, __setattr__ and __delattr__ to restrict access?

[Simon Sapin]

Le 24/02/2012 00:34, Victor Stinner a écrit :
> I'm trying to create read-only objects using a "frozendict" class.
> frozendict is a read-only dict. I would like to use frozendict for the
> class dict using a metaclass, but type.__new__() expects a dict and
> creates a copy of the input dict.
>
> I would be nice to support custom dict type: OrderedDict and
> frozendict for example. It looks possible to patch CPython to
> implement this feature, but first I would like first to know your
> opinion about this idea:-)

Hi,

Combining ideas from other messages in this thread: would this work?

1. Inherit from frozendict
2. Define a __getattr__ that defers to frozendict.__getitem__
3. Use an empty __slots__ so that there is no "normal" instance attribute.

Thinking about it a bit more, it’s probably the same as having a normal
__dict__ and raising in __setattr__ and __delattr__. Isn’t this how you
implement frozendict? (Raise in __setitem__, __delitem__, update, etc.)

Regards,
--
Simon Sapin

[David Townshend]

Using frozendict, and especially inheriting from it sounds unnecessarily complicated to me.  A simple class which doesn't allow changes to instance attributes could be implemented something like this:

class Three_inst:

    

    @property

    def value(self):

        return 3

    def __setattr__(self, attr, value):

        raise AttributeError

    def __delattr__(self, attr):

        raise AttributeError

Or, if you're worried about changes to the class attributes, you could do basically the same thing using a metaclass (python 2.7 syntax):

class FinalMeta(type):

    def __setattr__(cls, attr, value):

        if attr in cls.__dict__ or '__done__' in cls.__dict__:

            raise AttributeError

        else:

            type.__setattr__(cls, attr, value)

    def __delattr__(cls, attr):

        raise AttributeError

class Three:

    __metaclass__ = FinalMeta

    value = 3

    __done__ = True   # There may be a neater way to do this...

Each of the following examples will fail:

>>> Three.another_value = 4

>>> Three.value = 4

>>> del Three.value

>>> three = Three(); three.value = 4

Actually, I think this is quite a nice illustration of what can be done with metaclasses!

David

[Victor Stinner]

type.__setattr__(Three, 'value', 4) changes the value.

Victor

> class FinalMeta(type):
>
>     def __setattr__(cls, attr, value):
>         if attr in cls.__dict__ or '__done__' in cls.__dict__:
>             raise AttributeError
>         else:
>             type.__setattr__(cls, attr, value)
>
>     def __delattr__(cls, attr):
>         raise AttributeError
>
>
> class Three:
>     __metaclass__ = FinalMeta
>     value = 3
>     __done__ = True   # There may be a neater way to do this...
>
> Each of the following examples will fail:
>
>>>> Three.another_value = 4
>>>> Three.value = 4
>>>> del Three.value
>>>> three = Three(); three.value = 4

[David Townshend]

Ah, I think I misunderstood exactly what you were trying to achieve.  To me, that is essentially immutable - if I ever found myself using type.__setattr__ to change a variable I'd have to seriously question what I was doing!  But that would be a way around it, and I don't think it would be possible to implement it fully in python.  On the other hand, the same argument could be made for the introduction of private variables; Class.__var is not private because it can be changed through Class._Class__var.  I'd also consider having to do this to be indicative of a design flaw in my code.

[Simon Sapin]

Le 26/02/2012 14:56, Victor Stinner a écrit :
> type.__setattr__(Three, 'value', 4) changes the value.

Then there is the question of how much craziness you want to protect
from. Nothing is ever truly private or immutable in CPython, given
enough motivation and ctypes.

See for example Armin Ronacher’s "Bad Ideas" presentation, especially
the "Interpreter Warfare" part near the end:

https://ep2012.europython.eu/media/conference/slides/5-years-of-bad-ideas.pdf

I think that the code patching tracebacks is in production in Jinja2.
I’m sure frozensets could be modified in a similar way.

The point is: immutable data types protect against mistakes more than
someone truly determined to break the rules. With that in mind, I think
that having to go through __setattr__ is good enough to make sure it’s
not accidental.

Regards,
--
Simon Sapin

[David Townshend]

My point exactly!

[Victor Stinner]

>> type.__setattr__(Three, 'value', 4) changes the value.
>
> Then there is the question of how much craziness you want to protect from.
> Nothing is ever truly private or immutable in CPython, given enough
> motivation and ctypes.

My pysandbox project uses various hacks to secure Python. The attacker
doesn't care of writing pythonic code, (s)he just want to break the
sandbox :-) See my pysandbox project for more information:
https://github.com/haypo/pysandbox/

See sandbox/test/ if you like weird code :-) Tests ensure that the
sandbox is safe.

Constant types would also help optimization, especially PyPy JIT.

Victor

[Mark Janssen]

On Sat, Feb 25, 2012 at 4:05 PM, Steven D'Aprano <st...@pearwood.info> wrote:
> Ned Batchelder wrote:
>
>> The Python answer for people who want read-only data structures has always
>> been, "Don't modify them if you don't want to, and write docs that tell
>> other people not to as well."  What are you building that this answer isn't
>> good enough?
>
> That is silly. That alleged "Python answer" is like telling people that they
> don't need test frameworks or debuggers because the "Python answer" for
> people wanting to debug their code is not to write buggy code in the first
> place.

Perhaps a good middle ground for this is to NOT tie it to particular
data structures (like tuples vs lists), but abstract it by making an
"immutable bit" that is part of the basic Object type.  This doesn't
give complete security, but does *force* a choice by a human agent to
deliberately modify data.  (This was actually going to be implemented
in a sort of python fork several years ago.)  There could be a
"mutable?" check that returns True or False.

mark
Santa Fe, NM

[Rob Cliffe]

I suggested a "mutable" attribute some time ago.
This could lead to finally doing away with one of Python's FAQs: Why
does python have lists AND tuples? They could be unified into a single
type.
Rob Cliffe.

[Ethan Furman]

> I suggested a "mutable" attribute some time ago.
> This could lead to finally doing away with one of Python's FAQs: Why
> does python have lists AND tuples? They could be unified into a single
> type.

If a tuple is just an immutable list it will become worse with regards
to performance and memory space.

~Ethan~

[Mark Janssen]

On Mon, Feb 27, 2012 at 11:35 AM, Rob Cliffe <rob.c...@btinternet.com> wrote:
> I suggested a "mutable" attribute some time ago.
> This could lead to finally doing away with one of Python's FAQs: Why does
> python have lists AND tuples?  They could be unified into a single type.
> Rob Cliffe.

Yeah, that would be cool. It would force (ok, *allow*) the
documenting of any non-mutable attributes (i.e. when they're mutable,
and why they're being set immutable, etc.).

There an interesting question, then, should the mutable bit be on the
Object itself (the whole type) or in each instance....? There's
probably no "provable" or abstract answer to this, but rather just an
organization principle to the language....

m

[Mark Janssen]

On Mon, Feb 27, 2012 at 11:46 AM, Ethan Furman <et...@stoneleaf.us> wrote:
> If a tuple is just an immutable list it will become worse with regards to
> performance and memory space.

That's a good point also....

m

[Oleg Broytman]

On Mon, Feb 27, 2012 at 06:35:57PM +0000, Rob Cliffe wrote:
> I suggested a "mutable" attribute some time ago.
> This could lead to finally doing away with one of Python's FAQs: Why
> does python have lists AND tuples? They could be unified into a
> single type.

The main difference between lists and tuples is not mutability but
usage: lists are for a (unknown) number of similar items (a list of
messages, e.g.), tuples are for a (known) number of different items at
fixed positions (an address is a tuple of (country, city, street
address), for example).

Oleg.
--
Oleg Broytman http://phdru.name/ p...@phdru.name
Programmers don't die, they just GOSUB without RETURN.

[Victor Stinner]

2012/2/27 Oleg Broytman <p...@phdru.name>:

> On Mon, Feb 27, 2012 at 06:35:57PM +0000, Rob Cliffe wrote:
>> I suggested a "mutable" attribute some time ago.
>> This could lead to finally doing away with one of Python's FAQs: Why
>> does python have lists AND tuples?  They could be unified into a
>> single type.
>
>   The main difference between lists and tuples is not mutability but
> usage: lists are for a (unknown) number of similar items (a list of
> messages, e.g.), tuples are for a (known) number of different items at
> fixed positions (an address is a tuple of (country, city, street
> address), for example).

And tuple doesn't have append, extend, remove, ... methods.

Victor

[Mike Meyer]

On Mon, 27 Feb 2012 11:45:45 -0700
Mark Janssen <dreamin...@gmail.com> wrote:

> On Mon, Feb 27, 2012 at 11:35 AM, Rob Cliffe <rob.c...@btinternet.com> wrote:
> > I suggested a "mutable" attribute some time ago.
> > This could lead to finally doing away with one of Python's FAQs: Why does
> > python have lists AND tuples?  They could be unified into a single type.
> > Rob Cliffe.
> Yeah, that would be cool. It would force (ok, *allow*) the
> documenting of any non-mutable attributes (i.e. when they're mutable,
> and why they're being set immutable, etc.).

This also has implications for people working on making python
friendlier for concurrent and parallel programming.

> There an interesting question, then, should the mutable bit be on the
> Object itself (the whole type) or in each instance....? There's
> probably no "provable" or abstract answer to this, but rather just an
> organization principle to the language....

Ok, you said "non-mutable attributes" in the first paragraph. That to
me implies that the object bound to that attribute can't be
changed. This is different from the attribute being bound to an
immutable object, which this paragraph implies. Which do you want here?

<mike
--
Mike Meyer <m...@mired.org> http://www.mired.org/
Independent Software developer/SCM consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org

[Eric Snow]

On Mon, Feb 27, 2012 at 11:45 AM, Mark Janssen
<dreamin...@gmail.com> wrote:
> On Mon, Feb 27, 2012 at 11:35 AM, Rob Cliffe <rob.c...@btinternet.com> wrote:
>> I suggested a "mutable" attribute some time ago.
>> This could lead to finally doing away with one of Python's FAQs: Why does
>> python have lists AND tuples?  They could be unified into a single type.
>> Rob Cliffe.
>
> Yeah, that would be cool.  It would force (ok, *allow*) the
> documenting of any non-mutable attributes (i.e. when they're mutable,
> and why they're being set immutable, etc.).
>
> There an interesting question, then, should the mutable bit be on the
> Object itself (the whole type) or in each instance....?  There's
> probably no "provable" or abstract answer to this, but rather just an
> organization principle to the language....

In contrast to a flag on objects, one alternative is to have a
__mutable__() method for immutable types and __immutable__() for
mutable types. I'd be nervous about being able to make an immutable
object mutable at an arbitrary moment with the associated effect on
the hash of the object.

-eric

[Jim Jewett]

On Sat, Feb 25, 2012 at 6:32 PM, Masklinn <mask...@masklinn.net> wrote:
> On 2012-02-26, at 00:05 , Steven D'Aprano wrote:
>> - Immutable types can be used as keys in dicts.

Not always; for example, you can't use a tuple of lists, even though
the tuple itself is immutable.

> *technically*, you can use mutable types as dict keys if you define
> their __hash__ no? That is of course a bad idea when the instances
> are *expected* to be modified, but it should "work".

Not even a bad idea, if you define the hash carefully. (Similar to
java final.)

Once hash(obj) returns something other than -1, it should return that
same value forever. Attributes which do not contribute to the hash
can certainly still change.

That said, I would be nervous about changes to attributes that
contribute to __eq__, just because third party code may be so
surprised.

>>> class Str(str): pass
>>> a=Str("a")
>>> a.x=5
>>> a == "a"
True
>>> "x" in dir("a")
False
>>> "x" in dir(a)
True

-jJ

[Oleg Broytman]

On Mon, Feb 27, 2012 at 07:55:50PM +0100, Victor Stinner wrote:
> 2012/2/27 Oleg Broytman <p...@phdru.name>:
> > On Mon, Feb 27, 2012 at 06:35:57PM +0000, Rob Cliffe wrote:
> >> I suggested a "mutable" attribute some time ago.
> >> This could lead to finally doing away with one of Python's FAQs: Why
> >> does python have lists AND tuples?  They could be unified into a
> >> single type.
> >
> >   The main difference between lists and tuples is not mutability but
> > usage: lists are for a (unknown) number of similar items (a list of
> > messages, e.g.), tuples are for a (known) number of different items at
> > fixed positions (an address is a tuple of (country, city, street
> > address), for example).
>
> And tuple doesn't have append, extend, remove, ... methods.

Tuples are *also* read only, but being read only lists is not their
main purpose.

Oleg.
--
Oleg Broytman http://phdru.name/ p...@phdru.name
Programmers don't die, they just GOSUB without RETURN.

[Rob Cliffe]

On 27/02/2012 21:53, Oleg Broytman wrote:
> On Mon, Feb 27, 2012 at 07:55:50PM +0100, Victor Stinner wrote:
>> 2012/2/27 Oleg Broytman<p...@phdru.name>:
>>> On Mon, Feb 27, 2012 at 06:35:57PM +0000, Rob Cliffe wrote:
>>>> I suggested a "mutable" attribute some time ago.
>>>> This could lead to finally doing away with one of Python's FAQs: Why
>>>> does python have lists AND tuples? They could be unified into a
>>>> single type.
>>> The main difference between lists and tuples is not mutability but
>>> usage: lists are for a (unknown) number of similar items (a list of
>>> messages, e.g.), tuples are for a (known) number of different items at
>>> fixed positions (an address is a tuple of (country, city, street
>>> address), for example).
>> And tuple doesn't have append, extend, remove, ... methods.
> Tuples are *also* read only, but being read only lists is not their
> main purpose.
>
> Oleg.

With respect, I think you are thinking too narrowly, conditioned by
familiar usage.
Items of a list do not have to be similar (there is nothing in the
language that implies that).
And tuples are often - conceptually - extended, even though it actually
has to be done by building a new tuple - Python even allows you to write
tuple1 += tuple2
A unified type would have "mutating" methods such as append - it's just
that they would raise an error if the object's flag (however it was
implemented) defined it as immutable.

I visualised an actual object attribute, e.g. __mutable__, that could be
set to a boolean value. But having __mutable__() and __immutable__()
methods as suggested by Eric is an alternative. And there may well be
others.

Rob Cliffe

[Eric Snow]

On Mon, Feb 27, 2012 at 12:18 PM, Eric Snow <ericsnow...@gmail.com> wrote:
> On Mon, Feb 27, 2012 at 11:45 AM, Mark Janssen
> <dreamin...@gmail.com> wrote:
>> On Mon, Feb 27, 2012 at 11:35 AM, Rob Cliffe <rob.c...@btinternet.com> wrote:
>>> I suggested a "mutable" attribute some time ago.
>>> This could lead to finally doing away with one of Python's FAQs: Why does
>>> python have lists AND tuples?  They could be unified into a single type.
>>> Rob Cliffe.
>>
>> Yeah, that would be cool.  It would force (ok, *allow*) the
>> documenting of any non-mutable attributes (i.e. when they're mutable,
>> and why they're being set immutable, etc.).
>>
>> There an interesting question, then, should the mutable bit be on the
>> Object itself (the whole type) or in each instance....?  There's
>> probably no "provable" or abstract answer to this, but rather just an
>> organization principle to the language....
>
> In contrast to a flag on objects, one alternative is to have a
> __mutable__() method for immutable types and __immutable__() for
> mutable types.  I'd be nervous about being able to make an immutable
> object mutable at an arbitrary moment with the associated effect on
> the hash of the object.

Just to be clear, I meant that __mutable__() would return a mutable
version of the object, of a distinct mutable type, if the object
supported one. So for a tuple, it would return the corresponding
list. These would be distinct objects. Likewise obj.__immutable__()
would return a separate, immutable version of obj.

Such an approach could be applied to lists/tuples, sets/frozensets,
strings/bytearrays, bytes/bytearrays, and any other pairings we
already have. Unless a frozendict were added as a standard type, dict
would not have a match so an __immutable__() method would not be
added. In that case, trying to call dict.__immutable__() would be an
AttributeError, as happens now.

[Nick Coghlan]

On Tue, Feb 28, 2012 at 9:48 AM, Eric Snow <ericsnow...@gmail.com> wrote:
> Such an approach could be applied to lists/tuples, sets/frozensets,
> strings/bytearrays, bytes/bytearrays, and any other pairings we
> already have.  Unless a frozendict were added as a standard type, dict
> would not have a match so an __immutable__() method would not be
> added.  In that case, trying to call dict.__immutable__() would be an
> AttributeError, as happens now.

Folks, before retreading this ground, please make sure to review the
relevant past history and decide what (if anything) has changed since
Barry proposed the freeze protocol 5 years ago and the PEP was
rejected: http://www.python.org/dev/peps/pep-0351/

While hypergeneralisation of this behaviour is tempting, it really
isn't a solid abstraction. It's better to make use case specific
design decisions that handle all the corner cases relating to mutable
vs immutable variants of *particular* container types. The issues you
have to consider when converting a list to a tuple are not the same as
those that exist when converting bytearray to bytes or a set to a
frozenset.

Cheers,
Nick.

--
Nick Coghlan   |   ncog...@gmail.com   |   Brisbane, Australia

[Sven Marnach]

An easy way to create immutable instances is 'collections.namedtuple':

X = namedtuple("X", "a b")
x = X(a=4, b=2)
x.a + x.b # fine
x.a = 5 # AttributeError: can't set attribute
x.c = 5 # AttributeError: 'X' object has no attribute 'c'

Tricks using 'object.__setattr__()' etc. will fail since the instance
doesn't have a '__dict__'. The only data in the instance is stored in
a tuple, so it's as immutable as a tuple.

You can also derive from 'X' to add further methods. Remember to set
'__slots__' to an empty iterable to maintain immutability.

Cheers,
Sven

[Eric Snow]

On Mon, Feb 27, 2012 at 5:26 PM, Nick Coghlan <ncog...@gmail.com> wrote:
> On Tue, Feb 28, 2012 at 9:48 AM, Eric Snow <ericsnow...@gmail.com> wrote:
>> Such an approach could be applied to lists/tuples, sets/frozensets,
>> strings/bytearrays, bytes/bytearrays, and any other pairings we
>> already have.  Unless a frozendict were added as a standard type, dict
>> would not have a match so an __immutable__() method would not be
>> added.  In that case, trying to call dict.__immutable__() would be an
>> AttributeError, as happens now.
>
> Folks, before retreading this ground, please make sure to review the
> relevant past history and decide what (if anything) has changed since
> Barry proposed the freeze protocol 5 years ago and the PEP was
> rejected: http://www.python.org/dev/peps/pep-0351/
>
> While hypergeneralisation of this behaviour is tempting, it really
> isn't a solid abstraction. It's better to make use case specific
> design decisions that handle all the corner cases relating to mutable
> vs immutable variants of *particular* container types. The issues you
> have to consider when converting a list to a tuple are not the same as
> those that exist when converting bytearray to bytes or a set to a
> frozenset.

Point taken. :) I knew I'd heard the idea somewhere. I appreciate
how Raymond reacts here:

http://mail.python.org/pipermail/python-dev/2006-February/060802.html

and how Greg Ewing responds here:

http://mail.python.org/pipermail/python-dev/2006-February/060822.html

My point was that an __immutable__ flag was not a good idea. However,
I agree that the generic protocol is likewise inadvisable because it
fosters a generic design approach where a generic one is not
appropriate.

-eric