[Python-ideas] a simple namespace type

[Eric Snow]

Below I've included a pure Python implementation of a type that I wish
was a builtin. I know others have considered similar classes in the
past without any resulting change to Python, but I'd like to consider
it afresh[1][2].

class SimpleNamespace:
"""A simple attribute-based namespace."""
def __init__(self, **kwargs):
self.__dict__.update(kwargs) # or self.__dict__ = kwargs
def __repr__(self):
keys = sorted(k for k in self.__dict__ if not k.startswith('_'))
content = ("{}={!r}".format(k, self.__dict__[k]) for k, v in keys)
return "{}({})".format(type(self).__name__, ", ".join(content))

This is the sort of class that people implement all the time. There's
even a similar one in the argparse module, which inspired the second
class below[3]. If the builtin object type were dict-based rather
than slot based then this sort of namespace type would be mostly
superfluous. However, I also understand how that would add an
unnecessary resource burden on _all_ objects. So why not a new type?

Nick Coghlan had this objection recently to a similar proposal[4]:

Please, no. No new
just-like-a-namedtuple-except-you-can't-iterate-over-it type, and
definitely not one exposed in the collections module.

We've been over this before: collections.namedtuple *is* the standard
library's answer for structured records. TOOWTDI, and the way we have
already chosen includes iterability as one of its expected properties.

As you can see he's referring to "structured records", but I expect
that his objections could be extended somewhat to this proposal. I
see where he's coming from and agree relative to structured records.
However, I also think that a simple namespace type would be a benefit
to different use cases, namely where you want a simple dynamic
namespace.

Making a simple namespace class is trivial and likely just about
everyone has written one: "class Namespace: pass" or even
"type('Namespace', (), {})". Obviously the type in this proposal has
more meat, but that's certainly not necessary. So why a new type?

The main reason is that as a builtin type the simple namespace type
could be used in builtin modules[5][6][7].

Thoughts?

-eric


[1] http://mail.python.org/pipermail/python-dev/2012-May/119387.html
[2] http://mail.python.org/pipermail/python-dev/2012-May/119393.html
[3] http://hg.python.org/cpython/file/dff6c506c2f1/Lib/argparse.py#l1177
[4] http://mail.python.org/pipermail/python-dev/2012-May/119412.html
[5] http://mail.python.org/pipermail/python-dev/2012-May/119395.html
[6] http://mail.python.org/pipermail/python-dev/2012-May/119399.html
[7] http://mail.python.org/pipermail/python-dev/2012-May/119402.html

--------------------------

class Namespace(SimpleNamespace):
def __dir__(self):
return sorted(k for k in self.__dict__ if not k.startswith('_'))
def __eq__(self, other):
return self.__dict__ == other.__dict__
def __ne__(self, other):
return self.__dict__ != other.__dict__
def __contains__(self, name):
return name in self.__dict__
_______________________________________________
Python-ideas mailing list
Python...@python.org
http://mail.python.org/mailman/listinfo/python-ideas

[Yuval Greenfield]

[...]

I've implemented this a few times as well. I called it "AttributeDict" or "Record".

I think adding an __iter__ method would be beneficial. E.g.

class  SimpleNamespace :

     def __init__(self, **kwargs):

         self.__dict__.update(kwargs)  # or self.__dict__ = kwargs

         self.__iter__ = lambda: iter(kwargs.keys())

Why do we need this imo:

* sometimes x.something feels better than x['something']

* to ease duck-typing, making mocks, etc.

* Named tuple feels clunky for certain dynamic cases (why do I need to create the type for a one-off?)

I wonder if SimpleNameSpace should allow __getitem__ as well...

Yuval

[Eric Snow]

On Thu, May 24, 2012 at 5:59 AM, Yuval Greenfield <ubers...@gmail.com> wrote:
> On Tue, May 22, 2012 at 7:26 PM, Eric Snow <ericsnow...@gmail.com>
> wrote:
>>
>> Below I've included a pure Python implementation of a type that I wish
>> was a builtin.  I know others have considered similar classes in the
>> past without any resulting change to Python, but I'd like to consider
>> it afresh[1][2].

>> [...]
>
> I've implemented this a few times as well. I called it "AttributeDict" or
> "Record".
>
>
> I think adding an __iter__ method would be beneficial. E.g.
>
> class  SimpleNamespace :
>      def __init__(self, **kwargs):
>          self.__dict__.update(kwargs)  # or self.__dict__ = kwargs
>          self.__iter__ = lambda: iter(kwargs.keys())

I'd like to limit the syntactic overlap with dict as much as possible.
Effectively this is just a simple but distinct facade around dict to
give a namespace with attribute access. I suppose part of the
question is how much of the Mapping interface would belong instead to
a hypothetical Namespace interface. (I'm definitely _not_ proposing
such an unnecessary extra level of abstraction).

Regardless, if you want to do dict things then you can get the
underlying dict using vars(ns) or ns.__dict__ on your instance.
Alternately you can subclass the SimpleNamespace type to get all the
extra goodies you want, as I showed with the Namespace class at the
bottom of my first message.

> Why do we need this imo:
>
> * sometimes x.something feels better than x['something']
> * to ease duck-typing, making mocks, etc.
> * Named tuple feels clunky for certain dynamic cases (why do I need to
> create the type for a one-off?)

Yup.

> I wonder if SimpleNameSpace should allow __getitem__ as well...

Same thing: just use vars(ns) or a subclass of SimpleNamespace.

-eric

[Guido van Rossum]

On Thu, May 24, 2012 at 10:17 AM, Eric Snow <ericsnow...@gmail.com> wrote:
> On Thu, May 24, 2012 at 5:59 AM, Yuval Greenfield <ubers...@gmail.com> wrote:
>> On Tue, May 22, 2012 at 7:26 PM, Eric Snow <ericsnow...@gmail.com>
>> wrote:
>>>
>>> Below I've included a pure Python implementation of a type that I wish
>>> was a builtin.  I know others have considered similar classes in the
>>> past without any resulting change to Python, but I'd like to consider
>>> it afresh[1][2].
>>> [...]
>>
>> I've implemented this a few times as well. I called it "AttributeDict" or
>> "Record".

I tend to call it "Struct(ure)" -- I guess I like C better than Pascal. :-)

>> I think adding an __iter__ method would be beneficial. E.g.
>>
>> class  SimpleNamespace :
>>      def __init__(self, **kwargs):
>>          self.__dict__.update(kwargs)  # or self.__dict__ = kwargs
>>          self.__iter__ = lambda: iter(kwargs.keys())
>
> I'd like to limit the syntactic overlap with dict as much as possible.

+1

>  Effectively this is just a simple but distinct facade around dict to
> give a namespace with attribute access.  I suppose part of the
> question is how much of the Mapping interface would belong instead to
> a hypothetical Namespace interface. (I'm definitely _not_ proposing
> such an unnecessary extra level of abstraction).

Possibly there is a (weird?) parallel with namedtuple. The end result
is somewhat similar: you get to use attribute names instead of the
accessor syntax (x[y]) of the underlying type. But the "feel" of the
type is different, and inherits more of the underlying type
(namedtuple is immutable and has a fixed set of keys, whereas the type
proposed here is mutable and allows arbitrary keys as long as they
look like Python names).

> Regardless, if you want to do dict things then you can get the
> underlying dict using vars(ns) or ns.__dict__ on your instance.
> Alternately you can subclass the SimpleNamespace type to get all the
> extra goodies you want, as I showed with the Namespace class at the
> bottom of my first message.
>
>> Why do we need this imo:
>>
>> * sometimes x.something feels better than x['something']
>> * to ease duck-typing, making mocks, etc.
>> * Named tuple feels clunky for certain dynamic cases (why do I need to
>> create the type for a one-off?)
>
> Yup.
>
>> I wonder if SimpleNameSpace should allow __getitem__ as well...
>
> Same thing: just use vars(ns) or a subclass of SimpleNamespace.

--
--Guido van Rossum (python.org/~guido)

[Eric Snow]

On Thu, May 24, 2012 at 12:14 PM, Guido van Rossum <gu...@python.org> wrote:
> On Thu, May 24, 2012 at 10:17 AM, Eric Snow <ericsnow...@gmail.com> wrote:
>>  Effectively this is just a simple but distinct facade around dict to
>> give a namespace with attribute access.  I suppose part of the
>> question is how much of the Mapping interface would belong instead to
>> a hypothetical Namespace interface. (I'm definitely _not_ proposing
>> such an unnecessary extra level of abstraction).
>
> Possibly there is a (weird?) parallel with namedtuple. The end result
> is somewhat similar: you get to use attribute names instead of the
> accessor syntax (x[y]) of the underlying type. But the "feel" of the
> type is different, and inherits more of the underlying type
> (namedtuple is immutable and has a fixed set of keys, whereas the type
> proposed here is mutable and allows arbitrary keys as long as they
> look like Python names).

Yeah, the feel is definitely different. I've been thinking about this
because of the code for sys.implementation. Using a structseq would
probably been the simplest approach there, but a named tuple doesn't
feel right. In contrast, a SimpleNamespace would fit much better.

As far as this goes generally, the pattern of a simple, dynamic
attribute-based namespace has been implemented a zillion times (and
it's easy to do). This is because people find a simple dynamic
namespace really handy and they want the attribute-access interface
rather than a mapping.

In contrast, a namedtuple is, as Nick said, "the standard library's
answer for structured records". It's an immutable (attribute-based)
namespace implementing the Sequence interface. It's a tuple and
directly reflects the underlying concept of tuples in Python by giving
the values names.

SimpleNamespace (and the like) isn't a structured record. It's only
job is to be an attribute-based namespace with as simple an interface
as possible.

So why isn't a type like SimpleNamespace in the stdlib? Because it's
trivial to implement. There's a certain trivial-ness threshold a
function/type must pass before it gets canonized, and rightly so.

Anyway, while many would use something like SimpleNamespace out the
the standard library, my impetus was having it as a builtin type so I
could use it for sys.implementation. :)

FWIW, I have an implementation (pure Python + c extension) of
SimpleNamespace on PyPI:

http://pypi.python.org/pypi/simple_namespace

-eric

[Eric Snow]

Any further thoughts on this? Unless anyone is strongly opposed, I'd
like to push this forward.

[Calvin Spealman]

There is no good name for such a type. "Namespace" is a bad name, because
the term "namespace" is already a general term that describes a lot of things in
Python (and outside it) and shouldn't share a name with a specific
thing, this type.
That this specific type would also be within the more general namespace-concept
only makes that worse.

So, what do you call it?

Also, is this here because you don't like typing the square brackets
and quotes? If
so, does it only save you three characters and is that worth the increase to the
language size?

A final complaint against: would the existence of this fragment
python-learners education
to the point that they would defer learning and practicing to use
classes properly?

Sorry to complain, but someone needs to in python-ideas! ;-)
Calvin

> -eric
> _______________________________________________
> Python-ideas mailing list
> Python...@python.org
> http://mail.python.org/mailman/listinfo/python-ideas

--
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy

[Eric Snow]

On Sat, May 26, 2012 at 3:02 PM, Calvin Spealman <ironf...@gmail.com> wrote:
> On Sat, May 26, 2012 at 2:53 PM, Eric Snow <ericsnow...@gmail.com> wrote:
>> Any further thoughts on this?  Unless anyone is strongly opposed, I'd
>> like to push this forward.
>
> There is no good name for such a type. "Namespace" is a bad name, because
> the term "namespace" is already a general term that describes a lot of things in
> Python (and outside it) and shouldn't share a name with a specific
> thing, this type.
> That this specific type would also be within the more general namespace-concept
> only makes that worse.
>
> So, what do you call it?

Yeah, I've seen it called at least 10 different things. I'm certainly
open to whatever works best. I've called it "namespace" because it is
one of the two kinds of namespace in Python: mapping ([]-access) and
object (dotted-access). The builtin dict fills the one role and the
builtin object type almost fills the other. I guess
"dotted_namespace" or "attribute_namespace" would work if "namespace"
is too confusing.

> Also, is this here because you don't like typing the square brackets and quotes? If
> so, does it only save you three characters and is that worth the increase to the
> language size?

This is definitely the stick against which to measure!

It boils down to this: for me dotted-access communicates a different,
more stable sort of namespace than does []-access (a la dicts).
Certainly it is less typing, but that isn't really a draw for me.
Dotted access is a little easier to read, which is nice but not the
big deal for me. No, the big deal is the conceptual difference
inherent to access via string vs. access via identifier.

Though Python does not currently have a basic, dynamic,
attribute-based namespace type, it's trivial to make one: "class
Namespace: pass" or "type('Namespace', (), {})". While this has been
done countless times, it's so simple that no one has felt like it
belonged in the language. And I think that's fine, though it wouldn't
hurt to have something a little more than that (see my original
message).

So if it's so easy, why bother adding it? Well, "class Namespace:
pass" is not so simple to do using the C API. That's about it. (I
*do* think people would be glad to have a basic attribute-based
namespace type in the langauge.

> A final complaint against: would the existence of this fragment
> python-learners education
> to the point that they would defer learning and practicing to use
> classes properly?

This is an excellent point. I suppose it depends on who was teaching,
and how a new simple "namespace" type were exposed and documented. It
certainly is not a replacement for classes, which have much more
machinery surrounding state/methods/class-ness. If it made it harder
to learn Python then it would definitely have to bring *a lot* to the
table.

> Sorry to complain, but someone needs to in python-ideas! ;-)

Hey, I was more worried about the crickets I was hearing. :)

[Calvin Spealman]

This is probably the best case I've heard for such a type. Intent expression
is important!

The best names I was able to get crowdsourced from #python this morning
are:

- record
- flexobject
- attrobject
- attrdict
- nameddict
- namedobject

and the absolute worst name:
- Object

--
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy

[Sven Marnach]

Calvin Spealman schrieb am Sun, 27. May 2012, um 09:42:26 -0400:
> - record
> - flexobject
> - attrobject
> - attrdict
> - nameddict
> - namedobject

Since the proposed type is basically an `object` allowing attributes,
another option would be `attrobject`.

Adding an `__iter__()` method, as proposed earlier in this thread,
seems unnecessary; you can simply iterate over `vars(x)` for an
`attrobject` instance `x`.

Cheers,
Sven

[T.B.]

On 2012-05-27 19:08, Sven Marnach wrote:
> Calvin Spealman schrieb am Sun, 27. May 2012, um 09:42:26 -0400:
>> - record
>> - flexobject
>> - attrobject
>> - attrdict
>> - nameddict
>> - namedobject
>
> Since the proposed type is basically an `object` allowing attributes,
> another option would be `attrobject`.
>
> Adding an `__iter__()` method, as proposed earlier in this thread,
> seems unnecessary; you can simply iterate over `vars(x)` for an
> `attrobject` instance `x`.
>

Is this whole class really necessary? As said before, this type is
implemented numerous times:
* empty class (included in the Python Tutorial) [1]
* argparse.Namespace [2]
* multiprocessing.managers.Namespace [3]
* bunch (PyPI) that inherits from dict, instead of wrapping __dict__ [4]
* many more...

Each of them has a different semantics. Each is suited for a slightly
different use case and they are so easy to implement. So you can
customize to your liking - fields can or can't begin with "_", the later
__repr__ comment or the color of the shed. Still, it seems they do not
have a "killer feature" like namedtuple's efficiency.

Noticeable is how much they resemble a dict. Some let you iterate over
the keys, test for equality and even all of the builtin dict methods
(bunch). If you already use vars() for iteration, you might want a dict.

Funny that except for the easy "class Namespace: pass", the rest fail
repr for recursive/self-referential objects:

>>> from argparse/multiprocessing.managers/simplenamespace import Namespace
>>> ns = Namespace()
>>> ns.a = ns
>>> repr(ns)
...
RuntimeError: maximum recursion depth exceeded

The next snippet use the fact that dict's __repr__ knows how to handle
recursion to solve the RuntimeError problem:
def __repr__(self):
return "{}({!r})".format(self.__class__.__name__, self.__dict__)


TB

[1] http://docs.python.org/dev/tutorial/classes.html#odds-and-ends
[2] http://hg.python.org/cpython/file/c1eab1ef9c0b/Lib/argparse.py#l1177
[3]
http://hg.python.org/cpython/file/c1eab1ef9c0b/Lib/multiprocessing/managers.py#l913
[4] http://pypi.python.org/pypi/bunch

[Calvin Spealman]

On Sun, May 27, 2012 at 1:58 PM, T.B. <bauer...@gmail.com> wrote:
> On 2012-05-27 19:08, Sven Marnach wrote:
>> Calvin Spealman schrieb am Sun, 27. May 2012, um 09:42:26 -0400:
>>> - record
>>> - flexobject
>>> - attrobject
>>> - attrdict
>>> - nameddict
>>> - namedobject
>>
>> Since the proposed type is basically an `object` allowing attributes,
>> another option would be `attrobject`.
>>
>> Adding an `__iter__()` method, as proposed earlier in this thread,
>> seems unnecessary; you can simply iterate over `vars(x)` for an
>> `attrobject` instance `x`.
>>
>
> Is this whole class really necessary? As said before, this type is
> implemented numerous times:
> * empty class (included in the Python Tutorial) [1]
> * argparse.Namespace [2]
> * multiprocessing.managers.Namespace [3]
> * bunch (PyPI) that inherits from dict, instead of wrapping __dict__ [4]
> * many more...

All of the re-implementations of essentially the same thing is exactly why a
standard version is constantly suggested.

That said, it is so simple that it easily has many variants, because it is only
the base of the different ideas all these things implement.

--

Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy

[Eric V. Smith]

A test of the concept would be: could the uses of the similar classes in
the standard library be replaced with the proposed new implementation?

Eric.

[Oscar Benjamin]

On 27 May 2012 20:35, Eric V. Smith <er...@trueblade.com> wrote:

On 5/27/2012 3:31 PM, Calvin Spealman wrote:
> On Sun, May 27, 2012 at 1:58 PM, T.B. <bauer...@gmail.com> wrote:
>> On 2012-05-27 19:08, Sven Marnach wrote:
>>> Calvin Spealman schrieb am Sun, 27. May 2012, um 09:42:26 -0400:
>>>> - record
>>>> - flexobject
>>>> - attrobject
>>>> - attrdict
>>>> - nameddict
>>>> - namedobject
>>>
>>> Since the proposed type is basically an `object` allowing attributes,
>>> another option would be `attrobject`.
>>>
>>> Adding an `__iter__()` method, as proposed earlier in this thread,
>>> seems unnecessary; you can simply iterate over `vars(x)` for an
>>> `attrobject` instance `x`.

What about an `__iter__()` method that works like `dict.items()`? Then you can do a round trip with

    ns = attrobject(**d)

and

    d = dict(ns)

allowing you to quickly convert between attribute-based and item-based access in either direction.

[Nick Coghlan]

Slightly easier bar to reach: could the various incarnations be improved by using a new varobject type as a base class (e.g. I know I often use namedtuple as a base class rather than instantiating them directly, although I do the latter, too).

There's also a potentially less controversial alternative: just add an easy spelling for "type(name, (), {})" to the C API.

--
Sent from my phone, thus the relative brevity :)

[Eric Snow]

On Sun, May 27, 2012 at 2:09 PM, Nick Coghlan <ncog...@gmail.com> wrote:
> Slightly easier bar to reach: could the various incarnations be improved by
> using a new varobject type as a base class (e.g. I know I often use
> namedtuple as a base class rather than instantiating them directly, although
> I do the latter, too).

Good point. I do the same.

> There's also a potentially less controversial alternative: just add an easy
> spelling for "type(name, (), {})" to the C API.

I really like this. There's a lot of boilerplate to create just a
simple type like this in the C API. I'll see what I can come up with.
:)

As a namespace, it would be good to have a nice repr, but that's not a
show stopper.

-eric

[Eric Snow]

On Mon, May 28, 2012 at 10:34 AM, Eric Snow <ericsnow...@gmail.com> wrote:
> On Sun, May 27, 2012 at 2:09 PM, Nick Coghlan <ncog...@gmail.com> wrote:
>> There's also a potentially less controversial alternative: just add an easy
>> spelling for "type(name, (), {})" to the C API.
>
> I really like this.  There's a lot of boilerplate to create just a
> simple type like this in the C API.  I'll see what I can come up with.
>  :)

http://bugs.python.org/issue14942

[Jan Kaliszewski]

Eric Snow dixit (2012-05-28, 10:34):

> On Sun, May 27, 2012 at 2:09 PM, Nick Coghlan <ncog...@gmail.com> wrote:
> > Slightly easier bar to reach: could the various incarnations be improved by
> > using a new varobject type as a base class (e.g. I know I often use
> > namedtuple as a base class rather than instantiating them directly, although
> > I do the latter, too).
>
> Good point. I do the same.
>
> > There's also a potentially less controversial alternative: just add an easy
> > spelling for "type(name, (), {})" to the C API.
>
> I really like this. There's a lot of boilerplate to create just a
> simple type like this in the C API. I'll see what I can come up with.
> :)
>
> As a namespace, it would be good to have a nice repr, but that's not a
> show stopper.

Using classes as 'attribute containers' is suboptimal (which means that
in performance-critical parts of code you would have to implement a
namespace-like type anyway -- if you wanted to have attr-based syntax,
of course).

There should be one obvious way to do it. Now there is no one.

Cheers.
*j

[Eric Snow]

On Tue, May 29, 2012 at 11:34 AM, Jan Kaliszewski <z...@chopin.edu.pl> wrote:
> Using classes as 'attribute containers' is suboptimal (which means that
> in performance-critical parts of code you would have to implement a
> namespace-like type anyway -- if you wanted to have attr-based syntax,
> of course).

What are the performance problems of using a type object in this way?

>
> There should be one obvious way to do it. Now there is no one.

Yeah, I feel the same way.

-eric