I am fully aware of why vectors *can* be mutable and that having them start out life mutable makes sense (since you cannot really change an immutable vector back to mutable)
I didn't know that a fresh new vector is mutable. I have never used this feature. Why does it make sense that a new vector starts as mutable? Why a mutable vector is necessary?
In general, I am for that every output in Sage should be hashable and hence immutable by default, with exceptions explicitly declared as such.
On Thursday, 5 August 2021 at 18:42:55 UTC-7 Kwankyu Lee wrote:I didn't know that a fresh new vector is mutable. I have never used this feature. Why does it make sense that a new vector starts as mutable? Why a mutable vector is necessary?It can't start out immutable and then changed to mutable since ... well ... an immutable vector is immutable. There's a promise made that it won't change its content in its lifetime. We can't break that promise. So once you think mutable vectors should be an *option*, having them start out as mutable is an easy way of making them available.Mutable vectors and matrices are very important for numpy-style efficient algorithms: modifying matrices in-place (and I guess vectors as well) saves a lot of memory allocation churn for consecutively-allocated "float" and "int64" arrays. For our RealField and Integer/Rational based vectors this is already much less the case, because generally basic arithmetic *already* generates memory churn. I suspect in those cases a bit of pointer copying may not actually be so costly in comparison, so the overhead of newly allocating memory for results might actually not be so significant. In numpy and Pandas (which, oddly DOES copy a lot), it can be essential to do a lot of in-place modification, because the data structures can be huge, if you're working with high-dimensional vectors or matrices.
In general, I am for that every output in Sage should be hashable and hence immutable by default, with exceptions explicitly declared as such.So ... I suspect there's a lot of code in sage where immutable vectors wouldn't hurt, because a lot of operations will rely on the default arithmetic, which creates new vectors for results. However, there are some places where it would be a problem. Indeed, I think it would be quite reasonable to ask people to construct a vector explicitly as mutable if they need it.
So ... I suspect there's a lot of code in sage where immutable vectors wouldn't hurt, because a lot of operations will rely on the default arithmetic, which creates new vectors for results. However, there are some places where it would be a problem.
I looked into some. They use vectors like lists or mutable matrices.
On Thursday, 5 August 2021 at 21:01:28 UTC-7 Kwankyu Lee wrote:I looked into some. They use vectors like lists or mutable matrices.Yes, I expect that this will happen more with code in the future. People who have exposure to numpy et.al. tend to write code that way. I suspect that in many scenarios in sage, there's no benefit over working with a list except for some rare cases where one ALSO needs actual vector arithmetic.
But I'd expect the ship has sailed on this: there is too much code out there already that uses sage vectors as arrays to make it practical to change that default. So how bad would it be to potentially double the number of vector spaces in existence by giving them an extra parameter determining whether fresh vectors (such as those resulting from vector arithmetic) should be mutable or not.
--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/12f4e8a8-716e-4b0c-b26b-3ea708eb16f7n%40googlegroups.com.
So how bad would it be to potentially double the number of vector spaces in existence by giving them an extra parameter determining whether fresh vectors (such as those resulting from vector arithmetic) should be mutable or not.
Possible benefits may include:- potentially much faster: for similar reasons to numpy, a mutable sage vector can be much faster than a generic Python list.
Possible benefits may include:- potentially much faster: for similar reasons to numpy, a mutable sage vector can be much faster than a generic Python list.
- type safety: every element of a vector has the same parent ring; a list can have any elements in it
- length safety: you can't change the number of entries in a mutable list
To clarify, by "similar reasons to numpy", I meant that you open up
the possibility of using Cython, vectorized operations, JITs like
numba, etc. Some of these provide order of magnitude speedups and
aren't an option with generic Python lists.
--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/430ef312-156b-4364-8b73-456429d9decan%40googlegroups.com.
Generally, I would be in favor of having an option for immutable vectors by default, as I had similar uses before. Usually, I just convert everything to tuples.However, I am not sure if having an additional flag on the parent is worth the extra burden on maintenance. Already the sparsity flag is not always handled correctly (#29359, #29360), immutability may be even easier to miss.
In my own code, I tend to construct vectors by `vector(ring, [..])`, which relies on the uniqueness of the parent and will not preserve immutability of vectors. Though, it is fair to say that this code is already broken as it does not preserve sparsity either.
sage: a = vector(CDF, [CDF(1, i/(i+1)) for i in range(10^6)])sage: %time c1 = a.conjugate()Wall time: 4.39 ssage: %time c2 = a.parent()(a.numpy().conj())Wall time: 11.8 ms
I think Nils' original solution, of adding an immutable keyword option to the parent, is a good one. I've run into a similar issue with matrices, where I had to make them immutable before using them as dictionary keys.
I like the idea proposed in https://trac.sagemath.org/ticket/29101 (as posted by Matthias). Namely, introducing the option mutable=False for the element constructor. Then your code could be rewritten tosum( D[V(v0+w, mutable=False)] for w in W)which should be fine, I guess.
It may well be that it's convenient internally to have element constructors to allow for a "mutable=True/False" flag. It may even help with implementing the feature under consideration here: making sure that vector spaces have an option to produce (immediately) hashable elements as a result from arithmetic operations in that vector space. But it does not replace the need for being able to set that flag on a parent.
Have we considered the idea to simply call self.set_immutable() whenever
__hash__ is invoked? I think that would solve Nils' original problem
--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/20210810104118.6c1c6116%40l.
Have we considered the idea to simply call self.set_immutable() whenever
__hash__ is invoked?
On Tuesday, 10 August 2021 at 09:46:44 UTC-7 Matthias Koeppe wrote:On Monday, August 9, 2021 at 7:43:29 PM UTC-7 Lorenz Panny wrote:Have we considered the idea to simply call self.set_immutable() whenever
__hash__ is invoked?-1 on this; too complicated semanticsThe semantics are actually quite straightforward: it's hashability-on-demand.
The semantics are actually quite straightforward: it's hashability-on-demand.
I am -1 on hashability-on-demand for the suggested reason: it could lead to hard-to-track-down bugs in code where we end up calling a hash of something that could be *very* implicit, input to a UniqueRepresentation or a @cached_method/function. As Nils said, the user has no business hashing a mutable object, and we should explicitly tell the user that rather than silently changing it. It violates the principle of least surprise.
Imagine a bank that gave 50% of all your deposits to charity because it thought that is what you wanted to do because you checked out a charity website, but you only would find out about this when you checked your balance. I would not be happy with this bank.
--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/9a8aab3c-6b68-4b44-be63-c44bab94e12an%40googlegroups.com.
V=VectorSpace(GF(2),3)v = V.gen(1)w = V.gen(2)v.set_immutable() #this is annoying, but OKw.set_immutable() #this is again annoyingD={}D[v]=1D[w]=2D[v+w]=3 #the error here is uncalled for!
Just to add to all the ideas, what about using a with statement, eg:with Immutable():# the things Nils wishes were immutable are immutable by default here
It could even be refined to
with Immutable(V):
# elements of V are now immutable by default
But even then, some code elsewhere in sage might use the very same
parent (eg ZZ^2 is unique). This parent might want to deal with
mutable vectors. So toggling options on the parent does not look
like an option to me.
On Monday, 9 August 2021 at 15:54:03 UTC-7 Michael Jung wrote:I like the idea proposed in https://trac.sagemath.org/ticket/29101 (as posted by Matthias). Namely, introducing the option mutable=False for the element constructor. Then your code could be rewritten tosum( D[V(v0+w, mutable=False)] for w in W)which should be fine, I guess.No, that is not an improvement at all. imm(v0+w) is shorter than that.
The discussion of semantic innovations for the parents seems like a distraction to me. Note that in Python, the distinction between the mutable and immutable version of a type is usually expressed by different types: list/tuple, set/frozenset; and in libraries like SymPy, Matrix/ImmutableMatrix, ... Changing from mutable to immutable is done by creating a new object of the immutable version of the type. So the very operation "set_immutable" of Sage elements (mutating the mutability...) is already out of line with Python conventions; and in fact, this is what makes workaround functions like Nils' "imm" necessary if one wants to use functional notation in generator expressions etc.
Overall my suggestion would be to work out the intended semantics for the element constructor and element methods (copy and arithmetic), as I started in https://trac.sagemath.org/ticket/29101