How to equip a Symbol with an additional attribute?

[Carsten Knoll]

I want to equip an Symbol with an additional attribute to store some
specific information right in place.

For 'normal' Python classes it is no problem to dynamically create an
additional attribute for an already existing instance.

However, for sympy Symbols if I try

x = Symbol('x')
x.k = 0

I get the error

AttributeError: 'Symbol' object has no attribute 'k'

Is there any possibility to equip the Symbol x with this attribute?

I already tried this workarround:

class ExtendedSymbol(sp.Symbol):
pass


x = ExtendendSymbol('x')

x.k = 0 # no problem here


However then I have the following unwanted behavior:

x2 = sp.Symbol('x')
x2 == x # -> False

Any ideas?

Best regards,
Carsten.

[Amit Saha]

Not sure if it is possible for your situation, but may be you could
overrride the __eq__ method in your ExtendedSymbol class to call the
base class's method?

[Carsten Knoll]

I also thought in that direction, but how would that look like? I mean
overriding a method just to call the respective method of the base class
should be the same as not overiding the method at all, or not?

I then thought on mirroring the Symbol inside and piping all external
calls (including calls to '__eq__') to the wrapped Symbol instance and
fetching only AttributeErrors outside. But this seems to complicated.

The cleanest way would be to somehow dynamically highjack the
SymbolClass itself.

Equipping that class with new methods or class variables is no problem
but adding instance attributes is not possible. Not sure where this
behavior is implemented.

[Carsten Knoll]

On 05/18/2015 04:48 PM, Carsten Knoll wrote:
> On 05/18/2015 04:12 PM, Amit Saha wrote:
>> On Mon, May 18, 2015 at 11:56 PM, Carsten Knoll <Carste...@gmx.de> wrote:
>>> I want to equip an Symbol with an additional attribute to store some
>>> specific information right in place.

>

> Equipping that class with new methods or class variables is no problem
> but adding instance attributes is not possible. Not sure where this
> behavior is implemented.
>

If someone is interested: I found an (ugly) workaround:


# because sympy does not allow to dynamically attach attributes
# to symbols we set up our own infrastructure for storing them


import sympy as sp
sp._attribute_store = {}


def new_setattr(self, name, value):
try:
self.__orig_setattr__(name, value)
except AttributeError:
sp._attribute_store[(self, name)] = value


def new_getattr(self, name):
try:
res = self.__getattribute__(name)
except AttributeError, AE:
try:
res = sp._attribute_store[(self, name)]
except KeyError:
# raise the original AttributeError
raise AE
return res


sp.Symbol.__orig_setattr__ = sp.Symbol.__setattr__
sp.Symbol.__setattr__ = new_setattr

sp.Symbol.__getattr__ = new_getattr



Still, I would prefer to store the attribute directly 'inside' the
symbol instance but at least in the rest of my code this solution feel
like it were like that.

Carsten.

[Joachim Durchholz]

Am 18.05.2015 um 15:56 schrieb Carsten Knoll:
> I want to equip an Symbol with an additional attribute to store some
> specific information right in place.
>
> For 'normal' Python classes it is no problem to dynamically create an
> additional attribute for an already existing instance.
>
> However, for sympy Symbols if I try
>
> x = Symbol('x')
> x.k = 0
>
> I get the error
>
> AttributeError: 'Symbol' object has no attribute 'k'

That happens because we use __slots__ for performance reasons.
However, storing attributes in that way is a bad idea in general,
because you risk name conflicts with future versions of SymPy. (That's
also the reason why using subclasses to extend a library tends to break
over time.)

Alternative 1: You could set up a dict with a Symbol-to-attributes
mapping: Symbols are hashable (that's a guaranteed property).

Alternative 2: We could add a `userdict` attribute in Symbol, so that
SymPy users can store additional attributes there.

I'm not too happy with either alternative.
Maybe we can give you better help if you describe your use case in some
more detail.

Regards,
Jo

[Carsten Knoll]

>
> That happens because we use __slots__ for performance reasons.

I was not aware of the __slots__ mechanism. Now I know.

> However, storing attributes in that way is a bad idea in general,
> because you risk name conflicts with future versions of SymPy. (That's
> also the reason why using subclasses to extend a library tends to break
> over time.)

OK.

On the other hand syntax looks much cleaner

>
> Alternative 1: You could set up a dict with a Symbol-to-attributes
> mapping: Symbols are hashable (that's a guaranteed property).

Thats what I meanwhile did (see my other message on this thread). In
addition I redefine __setattr__ and __getatrr__ of the Symbol class to
hide the dictionary access behind the scene.

This feels somehow strange but it works. Lets see how long.

>
> Alternative 2: We could add a `userdict` attribute in Symbol, so that
> SymPy users can store additional attributes there.

This would be nice from my point of view. But wouldnt this cause
performance issues? As I understand, __slots__ is used to prevent every
Symbol instance from having a dict. But introducing a `userdict` would
negate this effort.

> I'm not too happy with either alternative.
> Maybe we can give you better help if you describe your use case in some
> more detail.

At some point I create symbols from which I know, that they are
derivatives and I want to store their order. Using the classes Function
and Derivative is no real option because I want the symbols to behave as
symbols on most occasions.

In general, I think there are situations, where it might be useful to
store some algorithm-specific extra information to some symbols.

Thanks and Regards,
Carsten.

[Aaron Meurer]

On Mon, May 18, 2015 at 12:29 PM, Joachim Durchholz <j...@durchholz.org> wrote:
> Am 18.05.2015 um 15:56 schrieb Carsten Knoll:
>>
>> I want to equip an Symbol with an additional attribute to store some
>> specific information right in place.
>>
>> For 'normal' Python classes it is no problem to dynamically create an
>> additional attribute for an already existing instance.
>>
>> However, for sympy Symbols if I try
>>
>> x = Symbol('x')
>> x.k = 0
>>
>> I get the error
>>
>> AttributeError: 'Symbol' object has no attribute 'k'
>
>
> That happens because we use __slots__ for performance reasons.
> However, storing attributes in that way is a bad idea in general, because
> you risk name conflicts with future versions of SymPy. (That's also the
> reason why using subclasses to extend a library tends to break over time.)

I'm not so sure about this.

The real reason that storing an attribute on x is a bad idea is that
SymPy implicitly assumes that it can replace one expression with the
same expression if the two are equal, so, e.g., if you had

x1 = Symbol('x')
x2 = Symbol('x')
x1.k = 0

and supposing it worked, along with x1 == x2 giving True, then SymPy
would assume that it has the freedom at any point to swap x1 with x2
and visa versa (and indeed, it does do this quite a bit with the
cache).

I think what you really want to do is create a subclass of Symbol that
stores the attributes in .args, so that they become part of the
equality of the object. Joachim's alternative 1 (store the
information separately) also sounds like a good one (and probably much
simpler to implement).

Aaron Meurer

>
> Alternative 1: You could set up a dict with a Symbol-to-attributes mapping:
> Symbols are hashable (that's a guaranteed property).
>
> Alternative 2: We could add a `userdict` attribute in Symbol, so that SymPy
> users can store additional attributes there.
>
> I'm not too happy with either alternative.
> Maybe we can give you better help if you describe your use case in some more
> detail.
>
> Regards,
> Jo
>

> --
> You received this message because you are subscribed to the Google Groups
> "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sympy+un...@googlegroups.com.
> To post to this group, send email to sy...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sympy.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sympy/555A2182.8070309%40durchholz.org.
>
> For more options, visit https://groups.google.com/d/optout.

[Joachim Durchholz]

Am 19.05.2015 um 00:44 schrieb Carsten Knoll:
>
>> That happens because we use __slots__ for performance reasons.
>
> I was not aware of the __slots__ mechanism. Now I know.
>
>> However, storing attributes in that way is a bad idea in general,
>> because you risk name conflicts with future versions of SymPy. (That's
>> also the reason why using subclasses to extend a library tends to break
>> over time.)
>
> OK.
>
> On the other hand syntax looks much cleaner

I prefer "maintainable" over "looks nice" any day.
However, that's just my professional bias because in software
engineering, maintenance dwarves all other concerns.

>> Alternative 1: You could set up a dict with a Symbol-to-attributes
>> mapping: Symbols are hashable (that's a guaranteed property).
>
> Thats what I meanwhile did (see my other message on this thread). In
> addition I redefine __setattr__ and __getatrr__ of the Symbol class to
> hide the dictionary access behind the scene.
>
> This feels somehow strange but it works. Lets see how long.

Expect some loss of performance with the __getattr__/__setattr__ overrides.
Not sure how much - could be large, could be negligible.

>> Alternative 2: We could add a `userdict` attribute in Symbol, so that
>> SymPy users can store additional attributes there.
>
> This would be nice from my point of view. But wouldnt this cause
> performance issues? As I understand, __slots__ is used to prevent every
> Symbol instance from having a dict. But introducing a `userdict` would
> negate this effort.

The mere existence of a dict does not impact SymPy much, it's a one-time
overhead during symbol creation. Unless you create a gazillion of
symbols, this should be negligible.

Still, I have seen that done (e.g. in JDK's Swing), and it comes with
issues.
E.g. two different third-party libraries fighting over the namespace
inside the `userdict`.
Also, all user attributes now being x.userdict.attr instead of x.attr.
Also, yet another detail to know about SymPy (as usual, there are alread
too many of them; it's adding to the learning curve).
Also, the constant temptation to use it for SymPy itself instead of
properly refactoring things.

An external dictionary isn't ideal but better than this, at least from
the maintainer's perspective.

>> Maybe we can give you better help if you describe your use case in some
>> more detail.
>
> At some point I create symbols from which I know, that they are
> derivatives and I want to store their order. Using the classes Function
> and Derivative is no real option because I want the symbols to behave as
> symbols on most occasions.

Ah. This sounds like a design problem in Derivative to me that we should
fix.
Can you post what you did, what you'd have needed to get out, and what
SymPy gave you?

> In general, I think there are situations, where it might be useful to
> store some algorithm-specific extra information to some symbols.

For algorithms of SymPy itself, I think the algorithms should be fixed :-)

It would be useful for coding external algorithms, but it does come with
ramifications (most importantly the potential for namespace conflicts);
from a maintenance perpective, it's far better if such an algorithm does
its own data management.

[Carsten Knoll]

On 05/19/2015 07:48 AM, Joachim Durchholz wrote:
>
> An external dictionary isn't ideal but better than this, at least from
> the maintainer's perspective.
>
>>> Maybe we can give you better help if you describe your use case in some
>>> more detail.
>>
>> At some point I create symbols from which I know, that they are
>> derivatives and I want to store their order. Using the classes Function
>> and Derivative is no real option because I want the symbols to behave as
>> symbols on most occasions.
>
> Ah. This sounds like a design problem in Derivative to me that we should
> fix.
> Can you post what you did, what you'd have needed to get out, and what
> SymPy gave you?

I think strongest argument for me was that the string representation of
the expressions gets too long for Functions and Derivatives. This
reduces my chance to "see" any structure.

Second argument is that I have a (own) library for modeling differential
forms which internally uses Symbol and does not straight forwardly work
with Function instances.

E.g.

x.name -> 'x'
x(t).name -> AttributeError

However, this discussions make me think, if using Functions would not be
the better alternative

>
>> In general, I think there are situations, where it might be useful to
>> store some algorithm-specific extra information to some symbols.
>
> For algorithms of SymPy itself, I think the algorithms should be fixed :-)
>
> It would be useful for coding external algorithms, but it does come with
> ramifications (most importantly the potential for namespace conflicts);
> from a maintenance perpective, it's far better if such an algorithm does
> its own data management.

OK. I can perfectly live with this.

[Carsten Knoll]

Thanks for pointing that out. I will add it to my tests.

>
> I think what you really want to do is create a subclass of Symbol that
> stores the attributes in .args, so that they become part of the
> equality of the object. Joachim's alternative 1 (store the
> information separately) also sounds like a good one (and probably much
> simpler to implement).

Subclassing and adding attributes in .args indeed seems more complicated.

I will stick with the external dict solution.


Regards,
Carsten.

[Aaron Meurer]

On Tue, May 19, 2015 at 4:41 AM, Carsten Knoll <Carste...@gmx.de> wrote:
> On 05/19/2015 07:48 AM, Joachim Durchholz wrote:
>>
>> An external dictionary isn't ideal but better than this, at least from
>> the maintainer's perspective.
>>
>>>> Maybe we can give you better help if you describe your use case in some
>>>> more detail.
>>>
>>> At some point I create symbols from which I know, that they are
>>> derivatives and I want to store their order. Using the classes Function
>>> and Derivative is no real option because I want the symbols to behave as
>>> symbols on most occasions.
>>
>> Ah. This sounds like a design problem in Derivative to me that we should
>> fix.
>> Can you post what you did, what you'd have needed to get out, and what
>> SymPy gave you?
>
> I think strongest argument for me was that the string representation of
> the expressions gets too long for Functions and Derivatives. This
> reduces my chance to "see" any structure.
>
> Second argument is that I have a (own) library for modeling differential
> forms which internally uses Symbol and does not straight forwardly work
> with Function instances.
>
> E.g.
>
> x.name -> 'x'
> x(t).name -> AttributeError

That's because x(t) is not really an instance of x. Symbol implements
__call__ so that x(t) returns a Function object (a completely
different class).

Aaron Meurer

>
> However, this discussions make me think, if using Functions would not be
> the better alternative
>
>>
>>> In general, I think there are situations, where it might be useful to
>>> store some algorithm-specific extra information to some symbols.
>>
>> For algorithms of SymPy itself, I think the algorithms should be fixed :-)
>>
>> It would be useful for coding external algorithms, but it does come with
>> ramifications (most importantly the potential for namespace conflicts);
>> from a maintenance perpective, it's far better if such an algorithm does
>> its own data management.
>
> OK. I can perfectly live with this.
>
>
>
>

> --
> You received this message because you are subscribed to the Google Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
> To post to this group, send email to sy...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sympy.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/555B0556.2040908%40gmx.de.

[G B]

Hi--

This seems as good a place as any to mention something I'd been trying to do some time ago where I ran into the same battle versus __slots__.  

I wanted to add a description attribute to my Symbols for documentation purposes.  I use Sympy almost exclusively inside of the IPython notebook for engineering work, and wind up with long sheets of derivations, substitutions, dead end attempts, etc.  I'm also trying to build up a personal library of formulas that I can import.

The problem I run in to is eventually, either due to complexity or time, I can't keep track of what all my symbols mean.  I like to keep the names themselves short so the symbolic output is easily understandable, but that relies on really strong conventions and a good memory to interpret-- which aren't always present...  =/

One solution was to put the description into the text of the notebook, which I often do, but then I have to find the place where that Symbol was defined to locate the descriptive text-- and then need to scroll back to the active cell to edit, then back and forth.

The solution I was after was to add a x.desc string to the Symbol, so I could just ask the Symbol itself what the hell I intended it to mean.  I then could modify Symbol to pretty print "F_{B}: the force of the bat".  If I got really energetic, I'd change Expr to pretty print:

"F_{B}=m_{b}*a_{b}

 Newton's Second Law of Motion where:

   F_{B}  is the force of the bat

   m_{b}  is the mass of the ball

   a_{b}   is the acceleration of the ball"

For all the maintenance reasons you describe, these are changes that should happen in the Sympy library itself, but it's easier for me to experiment in my own sandbox without upsetting the core code and the __slots__ prevents this.  There's also the chance that I can't convince enough people this should be a core part of the library, or to implement it in the way I'd like, and then I'm stuck.

Maintenance isn't quite as big an issue for me as I'm not building long standing applications-- I'm generating documents.  Sure, I'd like those to remain viable for as long as possible so I can edit them in the future, but it's not the same has having deployed a large codebase.  If I have to go through and change all my references from .desc to .desc2 for my notebook to run, it's not the end of the world.

Storing all of this in an outside data structure does have the potential advantage in that I could then look up a symbol if I knew what it meant but not its symbolic representation:

   SymbolIndex['accel ball']

or find all the formulas I have involving the ball:

   ExpressionIIndex['ball']

   ExpressionIndex[a_{b}]

but that would have required some deeper hacking into the Symbol creation and destruction mechanisms.

Anyway, probably wandered off topic a bit, but I just wanted to put a vote in for user attributes and the possible elimination of __slots__  if that isn't a significant performance hit.

[Joachim Durchholz]

Good points.
So I conclude that it would be nice if somebody came up with a way to
allow user-defined attributes without running into namespace conflicts.
Oh, and the mechanism should be easy to use for the casual user. I have
some ideas in that direction, but I'd like to see what others think first.

Independently of that, I'd like to see somebody check whether the
performace impact of __slots__ is still relevant; this may have changed
with newer Python versions and I'd like to have some contemporary
figures so we know what the actual trade-off is.