RFC: draft PEP for an infix matrix multiplication operator  Nathaniel Smith  3/8/14 6:22 PM  Hello SymPyoneers! Some of you may have already seen this, but I've been working on a draft PEP for adding a dedicated matrix multiplication operator to core Python:https://github.com/numpy/numpy/pull/4351 https://github.com/njsmith/numpy/blob/matmulpep/doc/neps/returnofrevengeofmatmulpep.rst I'd really appreciate comments and feedback from you guys  esp. since sympy is probably the most prominent library that *doesn't* currently follow the numpy convention of using * for elementwise multiplication. Does this seem like something that would benefit you? n 
Re: [sympy] RFC: draft PEP for an infix matrix multiplication operator  Aaron Meurer  3/8/14 11:17 PM  I've already alerted this list about it. You'll want to read the
discussion at https://groups.google.com/forum/#!searchin/sympy/infix/sympy/22w9ONLa7qo/7B8Vhks46hMJ. Aaron Meurer >  > You received this message because you are subscribed to the Google Groups > "sympy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sympy+un...@googlegroups.com. > To post to this group, send email to sy...@googlegroups.com. > Visit this group at http://groups.google.com/group/sympy. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sympy/f7b82624cb1347a6b9a4926edb5f8708%40googlegroups.com. > For more options, visit https://groups.google.com/d/optout. 
Re: [sympy] RFC: draft PEP for an infix matrix multiplication operator  Joachim Durchholz  3/8/14 11:50 PM  Am 09.03.2014 03:22, schrieb n...@vorpus.org:
> I'd really appreciate comments and feedback from you guys  esp. sinceNot really. SymPy already needs to overload all the standard operators to more closely follow mathematical conventions, while programmer conventions take a back seat. For that reason, Matrix * Matrix is matrix multiplication in SymPy, not elementwise multiplication  we have a Matrix type anyway, and we don't need to worry how the operator extends into arbitrary collections, so all the reasons for an elementwise multiplication don't apply. Also, from a mathematician's point of view, that's four(!) new operators. This looks like going overboard to resolve a rather specific use case. Yeah, for us, the "it's the one operator Python needs" holds no water at all. We want circumfix BraKet (i.e. <v and w>), we want nabla, we want cross product and scalar product, we want to be able to write x for the absolute value, etc. pp. So for SymPy, adding just one operator is solving the least of our problems, and adding a different one that covers a semantics we already have isn't solving any problems at all. That said, if you add new operators, we'll simply implement @ as a synonym to * for our Matrix classes, so we don't oppose either. Now personally, from a language designer's point of view, while I agree that matrix multiplication is an important operation, I disagree it's worth adding it. Python already has a multitude of operators, it's already hard to memorize the precedences by heart, and adding yet another operator is going to make things worse, not better. (The problem multiplies if you work with multiple programming languages; it's already a real problem for me.) Also, I'm missing a serious(!) discussion of the downsides of adding @ in the PEP. Reading it left a taste of "yeah but did they think about any potential problems and thought about ways to address them". From my perspective, the way to go isn't adding yet another operator, it's adding userdefined operators to Python. Yes you'd have to provide good answers to the BDFL's concerns, but I consider that a Good Thing :) Just my 2c. Regards, Jo 
Re: [sympy] RFC: draft PEP for an infix matrix multiplication operator  Joachim Durchholz  3/8/14 11:54 PM  One criticism on your PR.
The claim is that @ is a majority vote of relevant projects, but when I look at the actual list of projects, I'm seeing that 60% of the projects still need to implement @. That's quite far from a majority vote, and you're not listing projects that oppose the idea. Also, majority isn't a good yardstick for language design; I'd prefer if that argument were put into perspective. 
Re: [sympy] RFC: draft PEP for an infix matrix multiplication operator  Christophe Bal  3/9/14 1:09 AM  Hello, in the draft PEP, I can see : In numerical code, there are two important operations which compete for use of the * operator: elementwise multiplication, and matrix multiplication. Most Python code uses the * operator for the former, leaving no operator for matrix multiplication. Matrix multiplication is uniquely deserving of a new, dedicated infix operator
Sorry to say that but I think that this is really a wrong use of *. Why ? Just use SageMath, or open any math book and you will see that * is the matrix multiplication. You can't see "Most of project", that sounds like lobbying. Please, don't forget the math users !
For my part, I really think that it could be better to use a symmetric approach.
A math user of Sympy, Numpy and Scipy that will note appreciate to use @ for the matrix multiplication. Christophe BAL

Re: [sympy] RFC: draft PEP for an infix matrix multiplication operator  Joachim Durchholz  3/9/14 8:17 AM  Am 09.03.2014 10:09, schrieb Christophe Bal:
> Sorry to say that but I think that this is really a wrong use of *. Why ? *Just > use SageMath <http://www.sagemath.org/doc/tutorial/tour_linalg.html>, > or open any math book* and you will see that * is the matrix > multiplication. You can't see "Most of project", that sounds like lobbying.I guess you meant to direct this to njs :) > 1. Matrix multiplication will use * as it is done in math. > 2. Element wise multiplication can use something like ° that could be > better because it is near from the dot scalar of two vectors. *I do not > think that @ is really intuitive. * >Well, if we pushed for a scalar product in Python, we should push for "proper math notation", i.e. <x, y>. I wouldn't push for °. Not everybody can type it easily on their keyboard, plus people wouldn't expect it to be used for that (they'd expect a temperature or an angle, or a placeholder for an arbitrary operator if doing group theory). The question, however, is: Would anybody here be willing and able to invest the effort required to push for any language change PEP? I could set up the language design, but I lack the time to implement the proof of concept, and my Python experience isn't good enough to judge all the ramifications. If there are no volunteers, I guess we shouldn't be doing PEPs. 
Re: [sympy] RFC: draft PEP for an infix matrix multiplication operator  Christophe Bal  3/9/14 2:29 PM  Hello. I talk about the draft of the PEP. Joachim Durchholz, you're right for my symbol °, maybe @ could do the job. Implementing <x, y> could be painful regarding to the way the parser of the Python codes is done. I think that implementing @ in a similar way to * is less, or maybe not, painful.
On the other hand, we also have to heard that the elementwise multipication is often used.
The most important thing will be to have a good choice.
Best regards.
Christophe BAL 
Re: [sympy] RFC: draft PEP for an infix matrix multiplication operator  Joachim Durchholz  3/9/14 3:48 PM  Am 09.03.2014 22:29, schrieb Christophe Bal:
> On the other hand, we also have to heard that the elementwise multipicationActually, the PEP page says that only two out of five projects have it even implemented, let alone "often used". Are there even use cases for elementwise multiplication, apart from scalar product? Is the scalar product applicable to anything beyond vectors? 
Re: [sympy] RFC: draft PEP for an infix matrix multiplication operator  Christophe Bal  3/10/14 2:46 AM  Hello, when reading the draft, I've seen Elementwise multiplication is useful because it fits the common pattern for numerical code: it lets us easily and quickly perform a basic operation (scalar multiplication) on a large number of aligned values without writing a slow and cumbersome for loop. And this fits into a very general schema; e.g., in numpy, all Python operators work elementwise on arrays of all dimensionalities. Matrix multiplication is slightly more specialpurpose  it's only defined on 2d arrays, also known as "matrices"  but still used very heavily across all application areas; mathematically, it's one of the most fundamental operations there is. This is also the case in math so I do not think that it is a good argument. I really think that @ instead of * for scalar product is the good choice ! Don't forget the scientific math world which use more and more Python.
Christophe BAL 
Re: [sympy] RFC: draft PEP for an infix matrix multiplication operator  Joachim Durchholz  3/10/14 4:24 AM  Am 10.03.2014 10:46, schrieb Christophe Bal:
> Hello,Oh. Cumbersome... as in writing a single scalar multiplication. Plus you still need a "slow and cumbersome" loop to add the products, and if you use @ to create the elementwise products you also allocate an array that you don't need if you code it in a Python loop. And slow... well if speed is a concern, then maybe they should be doing numerics in C or Fortran. OMFG. The more I see the details, the less I like it. >> e.g., in numpy, *all Python operators work elementwise on arrays of all >> dimensionalities.* Sure. It just isn't a widespread operation  not by lines of code anyway. That's true. Heh. Sure  but then scientific math is still a small niche in the Python world. 
Re: [sympy] RFC: draft PEP for an infix matrix multiplication operator  Nathaniel Smith  3/10/14 5:30 AM 
I'm not sure what you mean here  can you elaborate? I've never seen a math book that defines "<<" on matrices to mean elementwiseleftbitshift, or "" as elementwisebitwiseor, or even "/" as elementwisedivision. These are all operations that numpy provides, though, and they're all useful and used in practical numbercrunching contexts.
Well, I'm not forgetting it, that's why I'm here :). But I also don't want to forget the Python scientific data processing world, which (a) is where these problems actually occur (because they make heavy use of both elementwise and matrix multiplication), (b) has, after a debate that's been going on for 15 years back to the early releases of Numeric, overwhelmingly settled on using * for elementwise multiplication, (c) involves something like 2030x more downstream users (judging by metrics like pypi downloads, github code search for "import numpy" versus "import sympy", number of pypi packages declaring a dependency on numpy versus sympy, etc.). So I just don't think I'm going to get very far if I go back to the numpy list and say "Alright, guys! We're switching * to mean matrix multiplication to make the mathematicians happy!". So the question is, what's the best system that balances between all these different communities and their constraints? Right now my feeling is that the PEP's proposal is the best available option, and if that means that sympy/sage will just ignore the PEP then that's fine, you guys are solving a different problem. But if you have a better way to balance these competing needs then I'd love to hear it! n 
Re: [sympy] RFC: draft PEP for an infix matrix multiplication operator  Joachim Durchholz  3/10/14 6:29 AM  Am 10.03.2014 13:30, schrieb n...@vorpus.org:
> > [...] I also don't want > to forget the Python scientific data processing world, which (a) is whereAck. I'm very, very sceptical, both of the claim (the PEP certainly does not make a strong case here) and of the usefulness of that. The only use case that I see for elementwise multiplication is the scalar product, which is a very weak case for @ because you still need the Python loop (or reduce() call) to add the products. Besides, the number of scalar product implementations you write is very limited  once per library, possibly with a handful of variants. Hardly enough to make "a clumsy loop" an argument of any weight. Prove me wrong if you can :) Well, you came here to get our perspective. No skin off my nose if you dislike it, or ignore it. I did mention that I'm rather sceptical about the reported "overwhelming support"; 2 out of 5 projects already implemented it, supposedly one of them is your own; 2 are reported as "planning to implement it". "Overwhelming support" and "general consensus" looks quite differently. And I hate being dragged into a bandwagon by promises of overwhelming support that has been generated as a selffulfilling prophecy; that's politics. So... no general consensus. In fact, we don't care much about the PEP, we live with * for multiplication just fine, and while we acknowledge the usefulness of matrix multiplication, we do think that it's no service to Python if every Python project promotes its set of pet operators  I bet the network guys could use some, generating output is another area (an operator for concatenating into an output stream would be more generally useful than matrix multiplication). Well, now that you're asking, I'll give my answer even if it's probably not what you want to read. Replace your math hat with a Python language designer hat. With that hat on, the question isn't just whether @ is going to help math and mathbased domains, it's whether it's going to help Python as a whole. Given that Python serves not just these domains, the relevance of matrix multiplication diminishes greatly. Personally, in Guido's place, I'd outright reject any domainspecific operator, and ask the people to come up with a better proposal for userdefined operators. What I have seen of userdefined operators discussion made me suspect that previous attempts were rejected not because userdefined operators, but because immature. Speaking of language design competence, I'm missing a serious discussion of syntactic ambiguities in your PEP. Saying "oh it's only used for annotations so no problem" is just handwaving, you can still run into ambiguities at the lexical level; also, there's no outline of the changes needed in the parser and how you can be sure it doesn't misinterpret an annotation for an operator or vice versa, and how the changes might the accuracy of error messages. This all in the sense of "do as you wish but that's the points I'd be asking if I were Guido, so you might want to clarify there before presenting this to the Python community at large". 
Re: [sympy] RFC: draft PEP for an infix matrix multiplication operator  Nathaniel Smith  3/10/14 7:40 AM  On Mon, Mar 10, 2014 at 1:29 PM, Joachim Durchholz <j...@durchholz.org> wrote:Noone uses elementwise multiplication to implement the scalar product, they just call np.dot or whatever the local equivalent is. Nonetheless, every single numeric language I know of has a dedicated 1or2character infix operator for elementwise multiplication; the only debate is whether it or matrix multiplication get more prominent billing. Pauli did some backoftheenvelope checks, which find that elementwise* is in fact called hundreds of times in the sklearn/nipy/scipy test suites, and this excludes scalar * matrix/scalar * vector cases: https://github.com/numpy/numpy/pull/4351#issuecomment37140164 It's at least in the same ballpark of usage as matrix multiplication. What I think you may be missing is that the whole "zen of arrays" is that they give you a coherent, composable system for writing simple scalar code that works over lots of scalars at once. You can write down a simple function like def f(x, y): return x * y + np.sin(x) and this looks like a nice simple scalar formula, you can test it by passing in scalars to x & y, BUT if you pass in arrays for x and y, then this same function will calculate that formula at thousands of values at once, and do it at nearC speeds. This is "composable" in the sense that f() now can be treated as a builtin "vectorized" (array) function, just like np.sin. And as a bonus, there's also a whole coherent system for doing more interesting manipulations, which f() will also participate in. E.g. if you want to evaluate this function on a grid, you can just do f(x_vals[:, np.newaxis], y_vals[np.newaxis, :]) which passes in 2d arrays for x and y with shape nby1 and 1byn, then this will give you the values at all combinations of x_vals and y_vals, returned as a neat nbyn 2d array, with reasonable memory use (e.g., x * y will allocate a 2d temporary, but sin(x) will allocate only a 1d temporary). Notice that this case uses elementwise* on 2d arrays. (If you want to see someone using trick who isn't me, then I basically just stole it from the official matplotlib 2d examples, e.g.: http://matplotlib.org/examples/images_contours_and_fields/streamplot_demo_features.html) Or, another example: I've recently been working on a project that involves calculating Bayesian posterior expectations using a simplified particle filtering method. If we have N particles (i.e., potential parameter settings for our statistical model), and we store the likelihoods of the data P(data  parameter value) in an array with N entries, the priors P(parameter value) in another array with N entries, and the values themselves in a third array, then in PEPnotation the posterior expectation of some function g() is: normalize(likelihoods * priors) @ g(values) (This is using @ in its scalarproduct guise, but in fact it turns out in the real code we have multiple different versions of this calculation that we have to do, so the different likelihoods are stored in a 2d array; the same formula works unchanged, though, with @ now acting as matrixvector multiplication.) The utility of elementwise multiplication just isn't a controversial point anywhere in the broader numeric ecosystem. Of course. I only bring it up because it's totally possible that there's some way of tweaking what's in the PEP to make it more useful to everybody, and if so then I hope you guys will help me think of it. Not sure what you're referring to here. Some projects that use * to mean elementwise multiplication include: numpy, theano, pycuda, pandas, numexpr, pytables, h5py, blaze, panda3d... The projects which use the opposite convention are overrepresented in the list in the PEP right now because I wanted to make sure to get feedback from other perspectives early, but at least scipy.sparse, pyoperators, and pyviennacl all reacted to the PEP by saying "hallelulah, this is what we really wanted all along" (and they all implement some way of doing elementwise multiplication, they just don't call it *). It's only sympy/sage that find elementwise multiplication so weird. And in any case, the point is really about what actual users do  numpy and the other projects listed there like theano and pandas collectively have ~hundreds of downstream projects that depend on them, and AFAICT well over 90% of those use np.ndarray exclusively, and ignore np.matrix to the maximum extent possible. pyoperators and pyviennacl, by contrast, are projects with ~1 user apiece. This is the "vast majority" referred to in the PEP. The day that I see 20% of PyCon tutorials are on the topic of "how to manipulate ipaddr objects", and there's a massive nonPython literature using a standard infix notation for manipulating ipaddrs, and they've spent 15 years struggling with the operators that Python already has, is the day that I'll support a new operator for the network guys too :). Right, this is the point of the sections called "But isn't matrix multiplication a pretty niche requirement?" and "So ``@`` is good for matrix formulas, but how common are those really?". I actually do feel that @ is more useful to Python as a whole than some of the existing operators. If you have any thoughts on how to make those sections stronger then I'd definitely be interested in hearing them. AFAICT his most recent comment on userdefined operators is: "I encourage the numpy folks to think about a language preprocessor that adds extra operators and perhaps other syntax." (https://plus.google.com/115212051037621986145/posts/hZVVtJ9bK3u) ...which doesn't give me much hope on this account. AFAICT the possibilities are: (1) no new operators, (2) one new very wellmotivated operator like @, (3) there is no possibility 3. I personally don't see how to make userdefined operators work in Python at all  among other issues, they would totally break the current compile/eval separation. I didn't go into details here because AFAICT they're trivial  am I missing something? Annotation@ is only legal as the first token in a statement; matmul@ is a binary operator, so it's only legal when *not* the first token in a statement. This is pretty unambiguous, and Python has lots of parsing issues that are more subtle than this ("is not", "if" in comprehensions, etc.). Indeed, and it's appreciated! n 
Re: [sympy] RFC: draft PEP for an infix matrix multiplication operator  Christophe Bal  3/10/14 8:33 AM  Sorry but in my last message, I wrote "I really think that @ instead of * for scalar product is the good choice ! " instead of "I really think that @ instead of * for element wise product is the good choice ! ".
So my proposition would be to switch the notations: the example of Nathaniel Smith normalize(likelihoods * priors) @ g(values) would become normalize(likelihoods @ priors) * g(values).
On the other hand, I heard the numeric community, sorry for this name, and your arguments are good too. The vectorization argument is a good one, and the fact that multiplications of matrix are less useful in most programs, the use of a weird symbol @ could be more adequate.
njs writes : "I've never seen a math book that defines "<<" on matrices to mean elementwiseleftbitshift, or "" as elementwisebitwiseor, or even "/" as elementwisedivision.".
Indeed, I just talk about +,  and /. Best regards. Christophe BAL 
Re: [sympy] RFC: draft PEP for an infix matrix multiplication operator  Joachim Durchholz  3/10/14 8:36 AM  Am 10.03.2014 15:40, schrieb Nathaniel Smith:
> Noone uses elementwise multiplication to implement the scalarIt's just that the scalar product is the only use case presented in the PEP. Seems like both are too useful to not implement them. Yes, that's useful from a programming perspective. It's just that this principle of "make scalar code work for arrays" carries you only so far, the principle breaks quite soon  for example in the form that multiplication simply isn't transferrable to vectors, let alone matrices, multiplication bifurcates and has multiple interpretations when going to scalars. Okay, accepted. Okay. From SymPy's perspective, being as close to mathematical formalisms as possible is good. So @ is bad because mathematicians don't write @ for elementwise multiplication  they use * (or juxtaposition) if in a context where elementwise multiplication is prominent and matrix multiplication isn't. Fortunately, we don't need to really oppose @ as long as we can support * for matrix multiplication, and I doubt we'll ever switch to * as elementwise multiplication simply because we's break existing code if we did. Networking is MUCH more than ip addresses :) But that was halfhumorous anyway. Doesn't mean that Python should repeat that mistake. It rejects that specific @ operator. It does not reject userdefined operators as a concept. Only if you do it wrongly. An operator definition is just syntactic sugar; essentially, for an operator #, a # b should be a shorthand for a.#(b). You need to assign precedence levels. These need to be tied to the operator symbols to make unambiguous parses  or at least the importing module needs to be able to override precedence levels. No or conflicting precedence levels > add parentheses, dammit, that's what they're for. The issue where I simply don't know how to best do it is the alphabet for operators.  What symbols can be operators? Only ASCII, or do we allow NLS characters like §?  Multicharacter operators?  If yes: Do we allow letters and digits in operators? E.g. any name with a + inside is an operator? I can't answer these because I'm not that far into Python's syntactic details. Another question is whether prefix and postfix operators should be allowed, too. Hm... well, I'd say it might close the road for other syntactic extensions in the future. One question that wasn't answered is whether asdf@ is the name asdf followed by the operator symbol @, or a single name asdf@. This might be answered by Python's syntax though. As I said, I'm not a syntax expert; maybe a link to background information on the parsing issues around @asannotation would help nonPythonsyntaxexperts understand the issues, and it would also demonstrate that you have indeed taken a good look at these issues. Sure. Doesn't mean that @ as an operator can't have its own subtleties. 