RFC: draft PEP for an infix matrix multiplication operator

[n...@vorpus.org]

Hello SymPyoneers!

Some of you may have already seen this, but I've been working on a draft PEP for adding a dedicated matrix multiplication operator to core Python:
   https://github.com/numpy/numpy/pull/4351
   https://github.com/njsmith/numpy/blob/matmul-pep/doc/neps/return-of-revenge-of-matmul-pep.rst

I'd really appreciate comments and feedback from you guys -- esp. since sympy is probably the most prominent library that *doesn't* currently follow the numpy convention of using * for elementwise multiplication. Does this seem like something that would benefit you?

-n

[Aaron Meurer]

I've already alerted this list about it. You'll want to read the
discussion at https://groups.google.com/forum/#!searchin/sympy/infix/sympy/22w9ONLa7qo/7B8Vhks46hMJ.

Aaron Meurer

> --
> You received this message because you are subscribed to the Google Groups
> "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sympy+un...@googlegroups.com.
> To post to this group, send email to sy...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sympy.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sympy/f7b82624-cb13-47a6-b9a4-926edb5f8708%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Joachim Durchholz]

Am 09.03.2014 03:22, schrieb n...@vorpus.org:
> I'd really appreciate comments and feedback from you guys -- esp. since
> sympy is probably the most prominent library that *doesn't* currently
> follow the numpy convention of using * for elementwise multiplication. Does
> this seem like something that would benefit you?

Not really. SymPy already needs to overload all the standard operators
to more closely follow mathematical conventions, while programmer
conventions take a back seat.
For that reason, Matrix * Matrix is matrix multiplication in SymPy, not
elementwise multiplication - we have a Matrix type anyway, and we don't
need to worry how the operator extends into arbitrary collections, so
all the reasons for an elementwise multiplication don't apply.

Also, from a mathematician's point of view, that's four(!) new
operators. This looks like going overboard to resolve a rather specific
use case.
Yeah, for us, the "it's the one operator Python needs" holds no water at
all. We want circumfix Bra-Ket (i.e. <v| and |w>), we want nabla, we
want cross product and scalar product, we want to be able to write |x|
for the absolute value, etc. pp.
So for SymPy, adding just one operator is solving the least of our
problems, and adding a different one that covers a semantics we already
have isn't solving any problems at all.

That said, if you add new operators, we'll simply implement @ as a
synonym to * for our Matrix classes, so we don't oppose either.


Now personally, from a language designer's point of view, while I agree
that matrix multiplication is an important operation, I disagree it's
worth adding it. Python already has a multitude of operators, it's
already hard to memorize the precedences by heart, and adding yet
another operator is going to make things worse, not better. (The problem
multiplies if you work with multiple programming languages; it's already
a real problem for me.)
Also, I'm missing a serious(!) discussion of the downsides of adding @
in the PEP. Reading it left a taste of "yeah but did they think about
any potential problems and thought about ways to address them".
From my perspective, the way to go isn't adding yet another operator,
it's adding user-defined operators to Python. Yes you'd have to provide
good answers to the BDFL's concerns, but I consider that a Good Thing :-)
Just my 2c.

Regards,
Jo

[Joachim Durchholz]

One criticism on your PR.

The claim is that @ is a majority vote of relevant projects, but when I
look at the actual list of projects, I'm seeing that 60% of the projects
still need to implement @.
That's quite far from a majority vote, and you're not listing projects
that oppose the idea.
Also, majority isn't a good yardstick for language design; I'd prefer if
that argument were put into perspective.

[Christophe Bal]

Hello,

in the draft PEP, I can see :

In numerical code, there are two important operations which compete for use of the * operator: elementwise multiplication, and matrix multiplication. Most Python code uses the * operator for the former, leaving no operator for matrix multiplication. Matrix multiplication is uniquely deserving of a new, dedicated infix operator 

Sorry to say that but I think that this is really a wrong use of *. Why ? Just use SageMath, or open any math book and you will see that * is the matrix multiplication. You can't see "Most of project", that sounds like lobbying. Please, don't forget the math users !

For my part, I really think that it could be better to use a symmetric approach.

1. Matrix multiplication will use * as it is done in math.
2. Element wise multiplication can use something like ° that could be better because it is near from the dot scalar of two vectors. I do not think that @ is really intuitive. 

A math user of Sympy, Numpy and Scipy that will note appreciate to use @ for the matrix multiplication.

Christophe BAL

--
You received this message because you are subscribed to the Google Groups "sympy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscribe@googlegroups.com.

To post to this group, send email to sy...@googlegroups.com.
Visit this group at http://groups.google.com/group/sympy.

To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/531C1E44.7040500%40durchholz.org.

[Joachim Durchholz]

Am 09.03.2014 10:09, schrieb Christophe Bal:
> Sorry to say that but I think that this is really a wrong use of *. Why ? *Just
> use SageMath <http://www.sagemath.org/doc/tutorial/tour_linalg.html>,
> or open any math book* and you will see that * is the matrix

> multiplication. You can't see "Most of project", that sounds like lobbying.
> Please, don't forget the math users !

I guess you meant to direct this to njs :-)

> For my part, I really think that it could be better to use a symmetric
> approach.
>

> 1. Matrix multiplication will use * as it is done in math.
> 2. Element wise multiplication can use something like ° that could be
> better because it is near from the dot scalar of two vectors. *I do not
> think that @ is really intuitive. *

>
> A math user of Sympy, Numpy and Scipy that will note appreciate to use @
> for the matrix multiplication.

Well, if we pushed for a scalar product in Python, we should push for
"proper math notation", i.e. <x, y>.

I wouldn't push for °. Not everybody can type it easily on their
keyboard, plus people wouldn't expect it to be used for that (they'd
expect a temperature or an angle, or a placeholder for an arbitrary
operator if doing group theory).

The question, however, is: Would anybody here be willing and able to
invest the effort required to push for any language change PEP?
I could set up the language design, but I lack the time to implement the
proof of concept, and my Python experience isn't good enough to judge
all the ramifications.
If there are no volunteers, I guess we shouldn't be doing PEPs.

[Christophe Bal]

Hello.

I talk about the draft of the PEP. Joachim Durchholz, you're right for my symbol °, maybe @ could do the job. Implementing <x, y> could be painful regarding to the way the parser of the Python codes is done. I think that implementing @ in a similar way to * is less, or maybe not, painful.

On the other hand, we also have to heard that the elementwise multipication is often used. 

The most important thing will be to have a good choice.

Best regards.

Christophe BAL

[Joachim Durchholz]

Am 09.03.2014 22:29, schrieb Christophe Bal:
> On the other hand, we also have to heard that the elementwise multipication
> is often used.

Actually, the PEP page says that only two out of five projects have it
even implemented, let alone "often used".

Are there even use cases for elementwise multiplication, apart from
scalar product?
Is the scalar product applicable to anything beyond vectors?

[Christophe Bal]

Hello,

when reading the draft, I've seen 

Elementwise multiplication is useful because it fits the common pattern for numerical code: it lets us easily and quickly perform a basic operation (scalar multiplication) on a large number of aligned values without writing a slow and cumbersome for loop. And this fits into a very general schema; e.g., in numpy, all Python operators work elementwise on arrays of all dimensionalities. Matrix multiplication is slightly more special-purpose -- it's only defined on 2d arrays, also known as "matrices" -- but still used very heavily across all application areas; mathematically, it's one of the most fundamental operations there is.

This is also the case in math so I do not think that it is a good argument. I really think that @ instead of * for scalar product is the good choice ! Don't forget the scientific math world which use more and more Python. 

Christophe BAL

[Joachim Durchholz]

Am 10.03.2014 10:46, schrieb Christophe Bal:
> Hello,
> when reading the draft, I've seen
>
>> Elementwise multiplication is useful because it fits the common pattern for
>> numerical code: it lets us easily and quickly perform a basic operation
>> (scalar multiplication) on a large number of aligned values without writing
>> a slow and cumbersome for loop.

Oh. Cumbersome... as in writing a single scalar multiplication.
Plus you still need a "slow and cumbersome" loop to add the products,
and if you use @ to create the elementwise products you also allocate an
array that you don't need if you code it in a Python loop.

And slow... well if speed is a concern, then maybe they should be doing
numerics in C or Fortran.

OMFG. The more I see the details, the less I like it.

>> And this fits into a very general schema;

>> e.g., in numpy, *all Python operators work elementwise on arrays of all
>> dimensionalities.*

Sure. It just isn't a widespread operation - not by lines of code anyway.

>> Matrix multiplication is slightly more special-purpose
>> -- it's only defined on 2d arrays, also known as "matrices" -- but still
>> used very heavily across all application areas; mathematically, it's one of
>> the most fundamental operations there is.

That's true.

> This is also the case in math so I do not think that it is a good argument.
> I really think that @ instead of * for scalar product is the good choice !

Heh.

> Don't forget the scientific math world which use more and more Python.

Sure - but then scientific math is still a small niche in the Python world.

[n...@vorpus.org]

On Monday, March 10, 2014 9:46:11 AM UTC, Christophe Bal wrote:

Hello,

when reading the draft, I've seen 

Elementwise multiplication is useful because it fits the common pattern for numerical code: it lets us easily and quickly perform a basic operation (scalar multiplication) on a large number of aligned values without writing a slow and cumbersome for loop. And this fits into a very general schema; e.g., in numpy, all Python operators work elementwise on arrays of all dimensionalities. Matrix multiplication is slightly more special-purpose -- it's only defined on 2d arrays, also known as "matrices" -- but still used very heavily across all application areas; mathematically, it's one of the most fundamental operations there is.

This is also the case in math so I do not think that it is a good argument.

I'm not sure what you mean here -- can you elaborate? I've never seen a math book that defines "<<" on matrices to mean elementwise-left-bitshift, or "|" as elementwise-bitwise-or, or even "/" as elementwise-division. These are all operations that numpy provides, though, and they're all useful and used in practical number-crunching contexts.
 

I really think that @ instead of * for scalar product is the good choice ! Don't forget the scientific math world which use more and more Python.

Well, I'm not forgetting it, that's why I'm here :-). But I also don't want to forget the Python scientific data processing world, which (a) is where these problems actually occur (because they make heavy use of both elementwise and matrix multiplication), (b) has, after a debate that's been going on for 15 years back to the early releases of Numeric, overwhelmingly settled on using * for elementwise multiplication, (c) involves something like 20-30x more downstream users (judging by metrics like pypi downloads, github code search for "import numpy" versus "import sympy", number of pypi packages declaring a dependency on numpy versus sympy, etc.). So I just don't think I'm going to get very far if I go back to the numpy list and say "Alright, guys! We're switching * to mean matrix multiplication to make the mathematicians happy!".

So the question is, what's the best system that balances between all these different communities and their constraints? Right now my feeling is that the PEP's proposal is the best available option, and if that means that sympy/sage will just ignore the PEP then that's fine, you guys are solving a different problem. But if you have a better way to balance these competing needs then I'd love to hear it!

-n

[Joachim Durchholz]

Am 10.03.2014 13:30, schrieb n...@vorpus.org:
>
> [...] I also don't want

> to forget the Python scientific data processing world, which (a) is where
> these problems actually occur (because they make heavy use of both
> elementwise and matrix multiplication),

Ack.

> (b) has, after a debate that's been
> going on for 15 years back to the early releases of Numeric, overwhelmingly
> settled on using * for elementwise multiplication,

I'm very, very sceptical, both of the claim (the PEP certainly does not
make a strong case here) and of the usefulness of that.

The only use case that I see for element-wise multiplication is the
scalar product, which is a very weak case for @ because you still need
the Python loop (or reduce() call) to add the products.
Besides, the number of scalar product implementations you write is very
limited - once per library, possibly with a handful of variants. Hardly
enough to make "a clumsy loop" an argument of any weight.

Prove me wrong if you can :-)

> (c) involves something
> like 20-30x more downstream users (judging by metrics like pypi downloads,
> github code search for "import numpy" versus "import sympy", number of pypi
> packages declaring a dependency on numpy versus sympy, etc.). So I just
> don't think I'm going to get very far if I go back to the numpy list and
> say "Alright, guys! We're switching * to mean matrix multiplication to make
> the mathematicians happy!".

Well, you came here to get our perspective.
No skin off my nose if you dislike it, or ignore it.

I did mention that I'm rather sceptical about the reported "overwhelming
support"; 2 out of 5 projects already implemented it, supposedly one of
them is your own; 2 are reported as "planning to implement it".
"Overwhelming support" and "general consensus" looks quite differently.
And I hate being dragged into a bandwagon by promises of overwhelming
support that has been generated as a self-fulfilling prophecy; that's
politics.

So... no general consensus. In fact, we don't care much about the PEP,
we live with * for multiplication just fine, and while we acknowledge
the usefulness of matrix multiplication, we do think that it's no
service to Python if every Python project promotes its set of pet
operators - I bet the network guys could use some, generating output is
another area (an operator for concatenating into an output stream would
be more generally useful than matrix multiplication).

> So the question is, what's the best system that balances between all these
> different communities and their constraints? Right now my feeling is that
> the PEP's proposal is the best available option, and if that means that
> sympy/sage will just ignore the PEP then that's fine, you guys are solving
> a different problem. But if you have a better way to balance these
> competing needs then I'd love to hear it!

Well, now that you're asking, I'll give my answer even if it's probably
not what you want to read.

Replace your math hat with a Python language designer hat.
With that hat on, the question isn't just whether @ is going to help
math and math-based domains, it's whether it's going to help Python as a
whole.
Given that Python serves not just these domains, the relevance of matrix
multiplication diminishes greatly. Personally, in Guido's place, I'd
outright reject any domain-specific operator, and ask the people to come
up with a better proposal for user-defined operators. What I have seen
of user-defined operators discussion made me suspect that previous
attempts were rejected not because user-defined operators, but because
immature.

Speaking of language design competence, I'm missing a serious discussion
of syntactic ambiguities in your PEP.
Saying "oh it's only used for annotations so no problem" is just
handwaving, you can still run into ambiguities at the lexical level;
also, there's no outline of the changes needed in the parser and how you
can be sure it doesn't misinterpret an annotation for an operator or
vice versa, and how the changes might the accuracy of error messages.

This all in the sense of "do as you wish but that's the points I'd be
asking if I were Guido, so you might want to clarify there before
presenting this to the Python community at large".

[Nathaniel Smith]

On Mon, Mar 10, 2014 at 1:29 PM, Joachim Durchholz <j...@durchholz.org> wrote:
> Am 10.03.2014 13:30, schrieb n...@vorpus.org:
>>
>>
>> [...] I also don't want
>>
>> to forget the Python scientific data processing world, which (a) is where
>> these problems actually occur (because they make heavy use of both
>> elementwise and matrix multiplication),
>
> Ack.
>
>> (b) has, after a debate that's been
>>
>> going on for 15 years back to the early releases of Numeric,
>> overwhelmingly
>> settled on using * for elementwise multiplication,
>
> I'm very, very sceptical, both of the claim (the PEP certainly does not make
> a strong case here) and of the usefulness of that.
>
> The only use case that I see for element-wise multiplication is the scalar
> product, which is a very weak case for @ because you still need the Python
> loop (or reduce() call) to add the products.
> Besides, the number of scalar product implementations you write is very
> limited - once per library, possibly with a handful of variants. Hardly
> enough to make "a clumsy loop" an argument of any weight.
>
> Prove me wrong if you can :-)

No-one uses elementwise multiplication to implement the scalar
product, they just call np.dot or whatever the local equivalent is.
Nonetheless, every single numeric language I know of has a dedicated
1-or-2-character infix operator for elementwise multiplication; the
only debate is whether it or matrix multiplication get more prominent
billing.

Pauli did some back-of-the-envelope checks, which find that
elementwise-* is in fact called hundreds of times in the
sklearn/nipy/scipy test suites, and this excludes scalar *
matrix/scalar * vector cases:
https://github.com/numpy/numpy/pull/4351#issuecomment-37140164
It's at least in the same ballpark of usage as matrix multiplication.

What I think you may be missing is that the whole "zen of arrays" is
that they give you a coherent, composable system for writing simple
scalar code that works over lots of scalars at once. You can write
down a simple function like

def f(x, y):
return x * y + np.sin(x)

and this looks like a nice simple scalar formula, you can test it by
passing in scalars to x & y, BUT if you pass in arrays for x and y,
then this same function will calculate that formula at thousands of
values at once, and do it at near-C speeds. This is "composable" in
the sense that f() now can be treated as a built-in "vectorized"
(array) function, just like np.sin. And as a bonus, there's also a
whole coherent system for doing more interesting manipulations, which
f() will also participate in. E.g. if you want to evaluate this
function on a grid, you can just do

f(x_vals[:, np.newaxis], y_vals[np.newaxis, :])

which passes in 2d arrays for x and y with shape n-by-1 and 1-by-n,
then this will give you the values at all combinations of x_vals and
y_vals, returned as a neat n-by-n 2d array, with reasonable memory use
(e.g., x * y will allocate a 2d temporary, but sin(x) will allocate
only a 1d temporary). Notice that this case uses elementwise-* on 2d
arrays.

(If you want to see someone using trick who isn't me, then I basically
just stole it from the official matplotlib 2d examples, e.g.:
http://matplotlib.org/examples/images_contours_and_fields/streamplot_demo_features.html)

Or, another example: I've recently been working on a project that
involves calculating Bayesian posterior expectations using a
simplified particle filtering method. If we have N particles (i.e.,
potential parameter settings for our statistical model), and we store
the likelihoods of the data P(data | parameter value) in an array with
N entries, the priors P(parameter value) in another array with N
entries, and the values themselves in a third array, then in
PEP-notation the posterior expectation of some function g() is:

normalize(likelihoods * priors) @ g(values)

(This is using @ in its scalar-product guise, but in fact it turns out
in the real code we have multiple different versions of this
calculation that we have to do, so the different likelihoods are
stored in a 2d array; the same formula works unchanged, though, with @
now acting as matrix-vector multiplication.)

The utility of elementwise multiplication just isn't a controversial
point anywhere in the broader numeric ecosystem.

>> (c) involves something
>>
>> like 20-30x more downstream users (judging by metrics like pypi downloads,
>> github code search for "import numpy" versus "import sympy", number of
>> pypi
>> packages declaring a dependency on numpy versus sympy, etc.). So I just
>> don't think I'm going to get very far if I go back to the numpy list and
>> say "Alright, guys! We're switching * to mean matrix multiplication to
>> make
>> the mathematicians happy!".
>
>
> Well, you came here to get our perspective.
> No skin off my nose if you dislike it, or ignore it.

Of course. I only bring it up because it's totally possible that
there's some way of tweaking what's in the PEP to make it more useful
to everybody, and if so then I hope you guys will help me think of it.

> I did mention that I'm rather sceptical about the reported "overwhelming
> support"; 2 out of 5 projects already implemented it, supposedly one of them
> is your own; 2 are reported as "planning to implement it".
> "Overwhelming support" and "general consensus" looks quite differently. And
> I hate being dragged into a bandwagon by promises of overwhelming support
> that has been generated as a self-fulfilling prophecy; that's politics.

Not sure what you're referring to here. Some projects that use * to
mean elementwise multiplication include: numpy, theano, pycuda,
pandas, numexpr, pytables, h5py, blaze, panda3d... The projects which
use the opposite convention are overrepresented in the list in the PEP
right now because I wanted to make sure to get feedback from other
perspectives early, but at least scipy.sparse, pyoperators, and
pyviennacl all reacted to the PEP by saying "hallelulah, this is what
we really wanted all along" (and they all implement some way of doing
elementwise multiplication, they just don't call it *). It's only
sympy/sage that find elementwise multiplication so weird.

And in any case, the point is really about what actual users do --
numpy and the other projects listed there like theano and pandas
collectively have ~hundreds of downstream projects that depend on
them, and AFAICT well over 90% of those use np.ndarray exclusively,
and ignore np.matrix to the maximum extent possible. pyoperators and
pyviennacl, by contrast, are projects with ~1 user apiece. This is the
"vast majority" referred to in the PEP.

> So... no general consensus. In fact, we don't care much about the PEP, we
> live with * for multiplication just fine, and while we acknowledge the
> usefulness of matrix multiplication, we do think that it's no service to
> Python if every Python project promotes its set of pet operators - I bet the
> network guys could use some, generating output is another area (an operator
> for concatenating into an output stream would be more generally useful than
> matrix multiplication).

The day that I see 20% of PyCon tutorials are on the topic of "how to
manipulate ipaddr objects", and there's a massive non-Python
literature using a standard infix notation for manipulating ipaddrs,
and they've spent 15 years struggling with the operators that Python
already has, is the day that I'll support a new operator for the
network guys too :-).

>> So the question is, what's the best system that balances between all these
>> different communities and their constraints? Right now my feeling is that
>> the PEP's proposal is the best available option, and if that means that
>> sympy/sage will just ignore the PEP then that's fine, you guys are solving
>> a different problem. But if you have a better way to balance these
>> competing needs then I'd love to hear it!
>
>
> Well, now that you're asking, I'll give my answer even if it's probably not
> what you want to read.
>
> Replace your math hat with a Python language designer hat.
> With that hat on, the question isn't just whether @ is going to help math
> and math-based domains, it's whether it's going to help Python as a whole.
> Given that Python serves not just these domains, the relevance of matrix
> multiplication diminishes greatly.

Right, this is the point of the sections called "But isn't matrix
multiplication a pretty niche requirement?" and "So ``@`` is good for
matrix formulas, but how common are those really?". I actually do feel
that @ is more useful to Python as a whole than some of the existing
operators. If you have any thoughts on how to make those sections
stronger then I'd definitely be interested in hearing them.

> Personally, in Guido's place, I'd
> outright reject any domain-specific operator, and ask the people to come up
> with a better proposal for user-defined operators. What I have seen of
> user-defined operators discussion made me suspect that previous attempts
> were rejected not because user-defined operators, but because immature.

AFAICT his most recent comment on user-defined operators is:
"I encourage the numpy folks to think about a language preprocessor
that adds extra operators and perhaps other syntax."
(https://plus.google.com/115212051037621986145/posts/hZVVtJ9bK3u)

...which doesn't give me much hope on this account. AFAICT the
possibilities are: (1) no new operators, (2) one new very
well-motivated operator like @, (3) there is no possibility 3.

I personally don't see how to make user-defined operators work in
Python at all -- among other issues, they would totally break the
current compile/eval separation.

> Speaking of language design competence, I'm missing a serious discussion of
> syntactic ambiguities in your PEP.
> Saying "oh it's only used for annotations so no problem" is just handwaving,
> you can still run into ambiguities at the lexical level; also, there's no
> outline of the changes needed in the parser and how you can be sure it
> doesn't misinterpret an annotation for an operator or vice versa, and how
> the changes might the accuracy of error messages.

I didn't go into details here because AFAICT they're trivial -- am I
missing something? Annotation-@ is only legal as the first token in a
statement; matmul-@ is a binary operator, so it's only legal when
*not* the first token in a statement. This is pretty unambiguous, and
Python has lots of parsing issues that are more subtle than this ("is
not", "if" in comprehensions, etc.).

> This all in the sense of "do as you wish but that's the points I'd be asking
> if I were Guido, so you might want to clarify there before presenting this
> to the Python community at large".

Indeed, and it's appreciated!

-n

[Christophe Bal]

Sorry but in my last message, I wrote "I really think that @ instead of * for scalar product is the good choice ! " instead of "I really think that @ instead of * for element wise product is the good choice ! ". 

So my proposition would be to switch the notations: the example of Nathaniel Smith normalize(likelihoods * priors) @ g(values) would become normalize(likelihoods @ priors) * g(values).

On the other hand, I heard the numeric community, sorry for this name, and your arguments are good too. The vectorization argument is a good one, and the fact that multiplications of matrix are less useful in most programs, the use of a weird symbol @ could be more adequate.

njs writes : "I've never seen a math book that defines "<<" on matrices to mean elementwise-left-bitshift, or "|" as elementwise-bitwise-or, or even "/" as elementwise-division.".

Indeed, I just talk about +, - and /.  

Best regards.

Christophe BAL

[Joachim Durchholz]

Am 10.03.2014 15:40, schrieb Nathaniel Smith:
> No-one uses elementwise multiplication to implement the scalar
> product, they just call np.dot or whatever the local equivalent is.

It's just that the scalar product is the only use case presented in the PEP.

> Nonetheless, every single numeric language I know of has a dedicated
> 1-or-2-character infix operator for elementwise multiplication; the
> only debate is whether it or matrix multiplication get more prominent
> billing.

Seems like both are too useful to not implement them.

> What I think you may be missing is that the whole "zen of arrays" is
> that they give you a coherent, composable system for writing simple
> scalar code that works over lots of scalars at once.

Yes, that's useful from a programming perspective.
It's just that this principle of "make scalar code work for arrays"
carries you only so far, the principle breaks quite soon - for example
in the form that multiplication simply isn't transferrable to vectors,
let alone matrices, multiplication bifurcates and has multiple
interpretations when going to scalars.

> The utility of elementwise multiplication just isn't a controversial
> point anywhere in the broader numeric ecosystem.

Okay, accepted.

> Of course. I only bring it up because it's totally possible that
> there's some way of tweaking what's in the PEP to make it more useful
> to everybody, and if so then I hope you guys will help me think of it.

Okay.

From SymPy's perspective, being as close to mathematical formalisms as
possible is good.
So @ is bad because mathematicians don't write @ for elementwise
multiplication - they use * (or juxtaposition) if in a context where
elementwise multiplication is prominent and matrix multiplication isn't.
Fortunately, we don't need to really oppose @ as long as we can support
* for matrix multiplication, and I doubt we'll ever switch to * as
elementwise multiplication simply because we's break existing code if we
did.

>> So... no general consensus. In fact, we don't care much about the PEP, we
>> live with * for multiplication just fine, and while we acknowledge the
>> usefulness of matrix multiplication, we do think that it's no service to
>> Python if every Python project promotes its set of pet operators - I bet the
>> network guys could use some, generating output is another area (an operator
>> for concatenating into an output stream would be more generally useful than
>> matrix multiplication).
>
> The day that I see 20% of PyCon tutorials are on the topic of "how to
> manipulate ipaddr objects",

Networking is MUCH more than ip addresses :-)

But that was half-humorous anyway.

> Right, this is the point of the sections called "But isn't matrix
> multiplication a pretty niche requirement?" and "So ``@`` is good for
> matrix formulas, but how common are those really?". I actually do feel
> that @ is more useful to Python as a whole than some of the existing
> operators.

Doesn't mean that Python should repeat that mistake.

>> Personally, in Guido's place, I'd
>> outright reject any domain-specific operator, and ask the people to come up
>> with a better proposal for user-defined operators. What I have seen of
>> user-defined operators discussion made me suspect that previous attempts
>> were rejected not because user-defined operators, but because immature.
>
> AFAICT his most recent comment on user-defined operators is:
> "I encourage the numpy folks to think about a language preprocessor
> that adds extra operators and perhaps other syntax."
> (https://plus.google.com/115212051037621986145/posts/hZVVtJ9bK3u)
>
> ...which doesn't give me much hope on this account. AFAICT the
> possibilities are: (1) no new operators, (2) one new very
> well-motivated operator like @, (3) there is no possibility 3.

It rejects that specific @ operator.
It does not reject user-defined operators as a concept.

> I personally don't see how to make user-defined operators work in
> Python at all -- among other issues, they would totally break the
> current compile/eval separation.

Only if you do it wrongly.

An operator definition is just syntactic sugar; essentially, for an
operator #, a # b should be a shorthand for a.#(b).

You need to assign precedence levels. These need to be tied to the
operator symbols to make unambiguous parses - or at least the importing
module needs to be able to override precedence levels.

No or conflicting precedence levels -> add parentheses, dammit, that's
what they're for.

The issue where I simply don't know how to best do it is the alphabet
for operators.
- What symbols can be operators? Only ASCII, or do we allow NLS
characters like §?
- Multi-character operators?
- If yes: Do we allow letters and digits in operators? E.g. any name
with a + inside is an operator?
I can't answer these because I'm not that far into Python's syntactic
details.

Another question is whether prefix and postfix operators should be
allowed, too.

>> Speaking of language design competence, I'm missing a serious discussion of
>> syntactic ambiguities in your PEP.
>> Saying "oh it's only used for annotations so no problem" is just handwaving,
>> you can still run into ambiguities at the lexical level; also, there's no
>> outline of the changes needed in the parser and how you can be sure it
>> doesn't misinterpret an annotation for an operator or vice versa, and how
>> the changes might the accuracy of error messages.
>
> I didn't go into details here because AFAICT they're trivial -- am I
> missing something? Annotation-@ is only legal as the first token in a
> statement; matmul-@ is a binary operator, so it's only legal when
> *not* the first token in a statement. This is pretty unambiguous,

Hm... well, I'd say it might close the road for other syntactic
extensions in the future.

One question that wasn't answered is whether asdf@ is the name asdf
followed by the operator symbol @, or a single name asdf@.
This might be answered by Python's syntax though. As I said, I'm not a
syntax expert; maybe a link to background information on the parsing
issues around @-as-annotation would help non-Python-syntax-experts
understand the issues, and it would also demonstrate that you have
indeed taken a good look at these issues.

> and
> Python has lots of parsing issues that are more subtle than this ("is
> not", "if" in comprehensions, etc.).

Sure.
Doesn't mean that @ as an operator can't have its own subtleties.