evaluate ImmutableMatrix by default

[krastano...@gmail.com]

This was already mentioned. Expressions of ImmutableMatrices are not
evaluated by default. A few people mentioned that they prefer
automatic evaluation. I also lean toward that choice. If there is
consensus on this topic I would open an issue for it.

Details:
The reason I want automatic evaluation is that I am working on some
hackish implementation of <some math object> and I have a ton of
```ImmutableMatrix(Matrix(instance_of_MatrixExpr))``` snippets. I need
ImmutableMatrix because of the implementation of __hash__. In another
thread I have posed a related question.

<some math object> in my case is quantum field theory tensors in Dirac
x Lorentz space

[Matthew Rocklin]

We should open an issue even if there isn't consensus. 

The current "don't evaluate by default" behavior was due to part of the conversation in this thread (Look for a post a fair ways down by Mateusz)

https://groups.google.com/d/topic/sympy/ekqyrxezwQ0/discussion

I think there should still be a way to create unevaluated MatMul's of ImmutableMatrices but that it should not be default. We've had far more requests recently for default evaluation than otherwise. 

Also, you should be able to turn ImmutableMatrix(Matrix(instance_of_MatrixExpr)) into just ImmutableMatrix(instance_of_MatrixExpr). 

Glad to see that they're being used.

--
You received this message because you are subscribed to the Google Groups "sympy" group.
To post to this group, send email to sy...@googlegroups.com.
To unsubscribe from this group, send email to sympy+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/sympy?hl=en.

[krastano...@gmail.com]

issue 3180 http://code.google.com/p/sympy/issues/detail?id=3180

> Also, you should be able to
> turn ImmutableMatrix(Matrix(instance_of_MatrixExpr)) into
> just ImmutableMatrix(instance_of_MatrixExpr).

Thanks for that. I was fairly sure I tried it but obviously I am mistaken.

[krastano...@gmail.com]

On a related note. At the moment mutability is contagious. I would
rather have it other way. Have I missed any discussions on this topic?

Also, sympify now works with ImmutableMatrices but it still fails on
MutableMatrices. Is there a fundamental reason behind that?

[krastano...@gmail.com]

> I think there should still be a way to create unevaluated MatMul's of
> ImmutableMatrices but that it should not be default. We've had far more
> requests recently for default evaluation than otherwise.

Something like this seems good enough:
In [9]: Add(1,2,evaluate=False)
Out[9]: 1 + 2

[Matthew Rocklin]

On a related note. At the moment mutability is contagious. I would
rather have it other way. Have I missed any discussions on this topic?

There weren't any serious discussions on the topic. It's easy to change in the matrices.classof function

 

Also, sympify now works with ImmutableMatrices but it still fails on
MutableMatrices. Is there a fundamental reason behind that?

It's not clear what sympify should do to a MutableMatrix? One option is to turn it into an ImmutableMatrix. If you do this then x*eye(3) isn't mutable because x will try to sympify it eye(3). 

[Ronan Lamy]

Le jeudi 22 mars 2012 à 22:52 +0100, krastano...@gmail.com a
écrit :

No, evaluate=False is a major source of pain. What you get out of it is
a malformed object that breaks lots of expectations. For instance, try
type(_9)(*_9.args)...

[krastano...@gmail.com]

> No, evaluate=False is a major source of pain. What you get out of it is
> a malformed object that breaks lots of expectations. For instance, try
> type(_9)(*_9.args)...

I would say that the problem is with the expectations, however I am
aware that this is not a constructive comment at the moment. I had
asked a few questions relate to this sometime ago and I will continue
the discussion there when I have the time. Thanks for making me aware
of this.

[krastano...@gmail.com]

>> Also, sympify now works with ImmutableMatrices but it still fails on
>> MutableMatrices. Is there a fundamental reason behind that?
>
>
> It's not clear what sympify should do to a MutableMatrix? One option is to
> turn it into an ImmutableMatrix. If you do this then x*eye(3) isn't mutable
> because x will try to sympify it eye(3).

Couldn't we have contagious immutability for matrix*matrix, contagious
mutability for symbol*matrix / matrix*symbol and sympification towards
immutable?

[krastano...@gmail.com]

> Couldn't we have contagious immutability for matrix*matrix, contagious
> mutability for symbol*matrix / matrix*symbol and sympification towards
> immutable?

It does not sound good. Basic*Mutable and Mutable*Basic should always
behave in the same way independently of what subclass of Basic we
have.

[Aaron Meurer]

To me, evaluation and immutability are completely separate. We should
have separate subclasses to do each (UnevaluatedImmutableMatrix is
probably better than ImmutableMatrix(evaluate=False)).

Aaron Meurer

[Matthew Rocklin]

-------------

Couldn't we have contagious immutability for matrix*matrix, contagious
mutability for symbol*matrix / matrix*symbol and sympification towards
immutable?

-------------

It does not sound good. Basic*Mutable and Mutable*Basic should always

behave in the same way independently of what subclass of Basic we
have

-------------

We have mechanisms to set policies for the following interactions

MatrixBase/MatrixBase interactions -- handled by "matrices.classof(A, B)"

Basic/Basic interactions -- handled by op_priority 

Basic/ImmutableMatrix -- works because ImmutableMatrix is Basic and has op_priority

Basic/MutableMatrix -- less easy - depends on Basic.__mul__(Mutable) raising NotImplementedError

So what happens when you type 

x = Symbol('x')

M = Matrix([[1,0], [0,1]])

x*M ? 

Python first tries x.__mul__(M). x doesn't recognize M so it tries M=sympify(M). This raises a NotImplementedError. 

Python then tries M.__rmul__(x) which does the usual (correct) thing because Matrices know how to deal with the situation.

Ideally we would like it to try M.__rmul__(x) first. This is usually handled by op_priority which selects which operand should determine the operation. Sadly op_priority only works for Basics. MutableMatrix is not Basic.

-------------

To me, evaluation and immutability are completely separate.  We should
have separate subclasses to do each (UnevaluatedImmutableMatrix is
probably better than ImmutableMatrix(evaluate=False)).

-------------

I agree that evaluation and immutability are separate. I think this is a perfect reason not to couple them with a new subclass however. I don't think we're talking about ImmutableMatrix(... , evaluate=False)), I think we're instead considering MatMul( ...  , evaluate=False). To me this seemed like the natural way to solve this problem until Ronan warned us away from it. 

Here are some thoughts. 

We could put some special logic into ImmutableMatrix.__add/mul__ to try to evaluate ImmutableMatrix*anything operations by default. We would fall back on creating MatMuls when we encountered other MatExprs that weren't ImmutableMatrices. We could use the explicit syntax MatMul(X, Y) if we wanted unevaluated ImmutableMatrix expressions. 

I think this solves our short term problem in a minimally disruptive way. However, I'm wary of it because we're building in one particular strategy at a low level. It would be nice to separate the construction of an expression with the simplification of it (this gets back to the AST and canonicalizers discussion). There are really two valid reduction strategies here. We don't currently have a mechanism in SymPy to deal with this other than to subclass ad naseum. But, despite this concern I think this is still probably the way to go for the short term. 

[Joachim Durchholz]

Am 23.03.2012 01:39, schrieb Aaron Meurer:
> To me, evaluation and immutability are completely separate.

Agreed.

> We should
> have separate subclasses to do each (UnevaluatedImmutableMatrix is
> probably better than ImmutableMatrix(evaluate=False)).

That will create a proliferation of classes. Mutable/Immutable and
Evaluated/Unevaluated is already four combinations; any additional
property would double the number of classes needed.

[Aaron Meurer]

Yeah, I was thinking about that too. I guess we should determine what
should be a separate class and what can just be a property of the
class. I think immutability should definitely be a separate class,
because this is a fundamental property in Python. I'm not sure about
evaluation. Ronan's pointed out some of the issues that can come from
making it a property.

Also, as Matthew noted, I guess evaluation should really be a property
of the container object, not the object itself. So we wouldn't really
need an UnevaluatedImmutibleMatrix, just an UnevaluatedMatrixMul.

By the way, I think that an unevaluated Matrix would have to be
immutable, because that's how Mul (and hence MatrixMul) works. So
there are really only three types of classes.

Aaron Meurer

[Joachim Durchholz]

Am 23.03.2012 10:27, schrieb Aaron Meurer:
> Yeah, I was thinking about that too. I guess we should determine what
> should be a separate class and what can just be a property of the
> class. I think immutability should definitely be a separate class,
> because this is a fundamental property in Python.

"Fundamental" is in the eye of the beholder.
There's a better litmus test: Do they have the same list of operations?
Require the operations the same preconditions, ensure the same
postconditions?
If yes, it's the same class. If no, it's a different class.

Of course, since we're *defining* the operations, we have considerable
leeway in making classes the same.
However, for (im)mutability, making the class interfaces the same does
not make any sense.

> I'm not sure about
> evaluation. Ronan's pointed out some of the issues that can come from
> making it a property.
>
> Also, as Matthew noted, I guess evaluation should really be a property
> of the container object, not the object itself. So we wouldn't really
> need an UnevaluatedImmutibleMatrix, just an UnevaluatedMatrixMul.

I'm not sure what exactly distinguishes an evaluated from an unevaluated
matrix, so I can't comment.

> By the way, I think that an unevaluated Matrix would have to be
> immutable, because that's how Mul (and hence MatrixMul) works. So
> there are really only three types of classes.

Sure, but the class proliferation will come soon enough.
Of course, sometimes the proliferation is worth the gains. Just make
sure it's really worth something.

[krastano...@gmail.com]

How about making MatExpr evaluate by default as it is the thing most
frequently requested at the moment and leaving the rest until the need
for it arises?

[Matthew Rocklin]

We need to have some way to store unevaluated immutable matrix multiplications. I use this on rare occasion, mostly for the latex printing. 

So we can do MatMul(..., evaluate=False) in this case but this is a bit misleading because I want the MatMul to simplify the expression in all ways except explicitly multiplying the ImmutableMatrices. 

Is there a problem with just having ImmutableMatrix.__mul__ look out for interactions with other ImmutableMatrices before creating a MatMul? This is a relatively clean short-circuit. This also maintains the separation between the expression and dense matrix sub-modules. I.e. it would be unfortunate to have checks for ImmutableMatrix in MatExpr land. We would have to cross sub-modules.

[Aaron Meurer]

On Sat, Mar 24, 2012 at 1:01 PM, Matthew Rocklin <mroc...@gmail.com> wrote:
> We need to have some way to store unevaluated immutable matrix
> multiplications. I use this on rare occasion, mostly for the latex
> printing.
>
> So we can do MatMul(..., evaluate=False) in this case but this is a bit
> misleading because I want the MatMul to simplify the expression in all ways
> except explicitly multiplying the ImmutableMatrices.
>
> Is there a problem with just having ImmutableMatrix.__mul__ look out for
> interactions with other ImmutableMatrices before creating a MatMul?

I don't understand what you mean here.

Aaron Meurer

[Alexey U. Gudchenko]

As I understand the main target to evaluate ImmutableMatrix by default
is to show the result of expression.

I agreed that it is better to see the result with in more convinient
form and key details.

Meanwhile at [1] was formulated arguments contra, and pro unevaluation
variant. I am warry why they are ignored.

Nevertheless, may be it is possible to join advantages of both variants
(non-evaluated by default and evaluated by default).

When I learn the sequences ans series I meet the similar problem, and
here are my suggestions:

For this we can just separate the task of showing of and the task
manipulation with expressions.

When I construct the expression (mul or add), I keep them in unevualeted
form as an expression, but every expression has `__getitem__` index,
which runs the calculation of desirable element (and, cache it).

If we work with such expressions in interactive mode, then printer
method form matrix used and it calls `__getitem__`, so we obtain the
readable answer.

Consider example (on example of ImmutableMatrix):

>>> a = ImmutableMatrix(3*eye(3))

# a <-- 3*ImmutableMatrix(...)

# here automatic simplification on the
# fly, which carring out common coeff.
# The "..." is written, because I don't know internal structure of the
# created object .

>>> b = 2*ImmutableMatrix(diag(1, 5, 6))

# b <-- 2*ImmutableMatrix(...)

>>> c = a*b

# c <-- 6*MatMul(ImmutableMatrix(...), ImmutableMatrix(...)

Note, so far we have not done any calculations with matrices.
No reasons for it.
We just construct the expression, simplify trivial things, if needed.

But if what to know (to show, or use in further calculations) then
result in readable form then we'll see:

>>> c[1:2]
30
>>> c
[6, 0, 0]
[0, 30, 0]
[0, 0, 36]

When we input "c[1:2]" then the
* `__getitem__ calls`
* the process of calculation of this element runs
(it is well-defined), which use only necessary calculations,
* then the result is cached,
* then it is returned and printed by printing system.

When we run "c", then (ordinary) printer for matrices is runs, which
also use __getitem__.

So, we can tune the printer system, and the expression behavior
independently during manipulation.

[1] http://code.google.com/p/sympy/issues/detail?id=3180

--
Alexey Gudchenko