Assigning part of indexed object
Fabrice P. Laussy
m[1] = Range[5];
is my indexed object (I could have m[2], etc.). Now I
want to do something like:
m[1][[1]] = 10;
with effect of having m[1] read:
{10, 2, 3, 4, 5}
but of course the line above evaluates to 1 = 10.
This could be achieved by having a non-indexed object get
in the way, e.g., m1=m[1], assign m1 and then return to
m[1]=m1, but I believe there should a more elegant way.
Bob Hanlon
m[1] = Range[5, 1, -1]
{5, 4, 3, 2, 1}
m1 = ReplacePart[m[1], 1 -> 10]
{10, 4, 3, 2, 1}
Bob Hanlon
---- "Fabrice P. Laussy" <fabrice...@n0spam-gmail.com> wrote:
=============
Heike Gramberg
m[1] = ReplacePart[m[1], 1 -> 10]
Heike
Bill Rowe
Laussy) wrote:
>How do you set the part of an indexed object, say:
>m[1] = Range[5];
>is my indexed object (I could have m[2], etc.). Now I want to do
>something like:
>m[1][[1]] = 10;
>with effect of having m[1] read:
>{10, 2, 3, 4, 5}
>but of course the line above evaluates to 1 = 10.
>This could be achieved by having a non-indexed object get in the
>way, e.g., m1=m[1], assign m1 and then return to m[1]=m1, but I
>believe there should a more elegant way.
The key is understanding where the list {1,2,3,4,5} is stored.
You can change m such that m[1][[1]] evaluates to 10 as follows:
In[1]:= m[1] = Range[5];
In[2]:= m[1]
Out[2]= {1,2,3,4,5}
In[3]:= DownValues[m] = ReplacePart[DownValues[m], {1, 2, 1} -> 10];
In[4]:= m[1]
Out[4]= {10,2,3,4,5}
magma
Use
Valeri Astanoff
My may to do it :
m[1] = Range[5];
m[1] = ReplacePart[m[1], 1 -> 10]
{10, 2, 3, 4, 5}
--
Valeri
Oleksandr Rasputinov
But it is an easy fix:
Unprotect[Set];
Set[sym_[[part_]], val_] := sym = ReplacePart[sym, part -> val];
Protect[Set];
In[65] := m[1] = Range[5]; m[1][[1]] = 10
Out[66] = {10,2,3,4,5}
Bear in mind that since this relies on pattern matching, it will not
work in situations
where pattern matching is not available via the interpreter (for
example, in compiled
code).
On Jun 22, 12:29 pm, "Fabrice P. Laussy" <fabrice.lau...@n0spam-
Fabrice P. Laussy
Thanks for the solution. Supplemented with Oleksandr Rasputinov's fix:
> Unprotect[Set];
> Set[sym_[[part_]], val_] := sym = ReplacePart[sym, part -> val];
> Protect[Set];
it works splendidly.
In "some elements of Mathematica Design (1992)" [http://goo.gl/nm1tT],
Stephan Wolfram makes a point where one should resist the temptation to
design functions to do things that look natural for them to do, as they
could break dramatically later on.
Here we have a clear case where Set seems it ought to work as redesigned by
Oleksandr. Is there any reason for this not being the default behaviour?
Leonid Shifrin
My guess is that the reasons are efficiency and immutability of expressions
in Mtahematica. By restricting Set
to variables or array elements, one can connect it directly to the internal
memory layout for that element. Note
that all proposed solutions are based on ReplacePart, which copies the
entire list to modify a single element.
OTOH, constructs like indexed variables are serviced by hash-tables
internally. Even if the value stored
in the hash table is a pointer to some array, that array will likely be
immutable. Whether it would be a
good idea to make it mutable I don't know, but the general ideology is that
expressions are immutable
in Mathematica. In other words, I don't think that the case of indexed
variables was overlooked when
designing Set - my guess is that the restriction to symbols is quite
intentional.
Regarding the "fix", I'd consider this not a fix but a recipe for disaster,
because modifying Set in such
a fashion is dangerous, not to mention that it will often be completely
unacceptable performance-wise. Let us say
you store a large array in a symbol and use a For loop with array indexing.
Suddenly all your array element assignments
become O(n) time, where n is the list size, or even worse for higher
dimensional tensors, not to mention increased
memory consumption. Here is an illustration:
In[18]:= a = Range[10000];
In[19]:= Unprotect[Set];
Set[sym_[[part_]], val_] := sym = ReplacePart[sym, part -> val];
Protect[Set];
In[27]:= Do[a[[i]] = a[[i]]^2, {i, Length[a]}] // Timing
Out[27]= {1.156, Null}
In[28]:= Unprotect[Set];
Clear[Set];
Protect[Set];
In[31]:= Do[a[[i]] = a[[i]]^2, {i, Length[a]}] // Timing
Out[31]= {0.047, Null}
You may do this for yourself if you wish (although I'd avoid even that),
but I'd strongly
advise against such modifications in production code or any code someone
else will be using. Set is one of the
most fundamental operations, and its modifications may have very far-
reaching and hard to predict consequences.
It may, for example, corrupt the system in very subtle ways, which you may
never be able to identify.
Unfortunately, the standard mechanism of "soft" system symbols overloading
via UpValues does not
quite work with Set (or at least I don't know of a way to make it work)
because Set holds its argument and UpValues
can only be defined for tags not deeper than level 1 in an expression. In
this post:
http://stackoverflow.com/questions/5886066/overloading-seta-b-a-b/5886322#5886322
I demonstrated a method, which I also don't like, but which is at least
safer because it tests for a
specific data type. Generally, the still safer way is to create your own
data type and introduce your own
assignment operator(s) - in this way, you don't have to modify Set at all.
Regards,
Leonid
Oleksandr Rasputinov
problem with ReplacePart as a direct substitute for Set:
m = Range[5];
ReplacePart[m, 1 -> 10] :> {10, 2, 3, 4, 5}
but
m[[1]] = 10 :> ( m = {10, 2, 3, 4, 5}; 10 ) :> 10
the following is thus a better solution than what I had
posted previously:
Unprotect[Set];
Set[sym_[[part_]], val_] := (
sym = ReplacePart[sym, part -> val];
sym[[part]]
);
Protect[Set];
Now:
Clear[m]; (* OwnValues will override DownValues *)
m[1] = Range[5];
m[1][[1]] = 10 :> ( m[1] = {10, 2, 3, 4, 5}; 10 ) :> 10
which is consistent with other uses of Set.
On Jun 24, 12:52 pm, "Fabrice P. Laussy" <fabrice.lau...@n0spam-
Oleksandr Rasputinov
I fully agree with all of your comments and of course you are right to
point out that modifying core functions such as Set is a bad idea in
general due to the likelihood of unintended consequences. Of course,
the performance issue introduced for setting parts of OwnValues can
be repaired to some extent:
indexedQ = Function[sym,
Symbol =!= Head@Unevaluated[sym] =!= Unevaluated[sym],
HoldFirst
];
Unprotect[Set];
Set[(sym_?indexedQ)[[part_]], val_] := (
sym = ReplacePart[sym, part -> val];
sym[[part]]
);
Protect[Set];
which restores the O(1) behaviour (albeit at the cost of an increased
constant factor). But even so, I will join with you in stressing that this
general approach is dangerous and should be attempted only with
extreme caution, if at all.
Best,
O. R.
On Jun 25, 10:30 am, Leonid Shifrin <lsh...@gmail.com> wrote:
> Hi Fabrice,
>
> My guess is that the reasons are efficiency and immutability of expres=
sions
> in Mtahematica. By restricting Set
> to variables or array elements, one can connect it directly to the intern=
al
> memory layout for that element. Note
> that all proposed solutions are based on ReplacePart, which copies the
> entire list to modify a single element.
> OTOH, constructs like indexed variables are serviced by hash-tables
> internally. Even if the value stored
> in the hash table is a pointer to some array, that array will likely be
> immutable. Whether it would be a
> good idea to make it mutable I don't know, but the general ideology is th=
at
> expressions are immutable
> in Mathematica. In other words, I don't think that the case of indexed
> variables was overlooked when
> designing Set - my guess is that the restriction to symbols is quite
> intentional.
>
> Regarding the "fix", I'd consider this not a fix but a recipe for disa=
ster,
> because modifying Set in such
> a fashion is dangerous, not to mention that it will often be completel=
y
> unacceptable performance-wise. Let us say
> you store a large array in a symbol and use a For loop with array indexin=
g.
> Suddenly all your array element assignments
> become O(n) time, where n is the list size, or even worse for higher
> dimensional tensors, not to mention increased
> memory consumption. Here is an illustration:
>
> In[18]:= a = Range[10000];
>
> In[19]:= Unprotect[Set];
> Set[sym_[[part_]], val_] := sym = ReplacePart[sym, part -> val];
> Protect[Set];
>
> In[27]:= Do[a[[i]] = a[[i]]^2, {i, Length[a]}] // Timing
>
> Out[27]= {1.156, Null}
>
> In[28]:= Unprotect[Set];
> Clear[Set];
> Protect[Set];
>
> In[31]:= Do[a[[i]] = a[[i]]^2, {i, Length[a]}] // Timing
>
> Out[31]= {0.047, Null}
>
> You may do this for yourself if you wish (although I'd avoid even that=
),
> but I'd strongly
> advise against such modifications in production code or any code someone
> else will be using. Set is one of the
> most fundamental operations, and its modifications may have very far-
> reaching and hard to predict consequences.
> It may, for example, corrupt the system in very subtle ways, which you ma=
y
> never be able to identify.
>
> Unfortunately, the standard mechanism of "soft" system symbols overloadin=
g
> via UpValues does not
> quite work with Set (or at least I don't know of a way to make it work)
> because Set holds its argument and UpValues
> can only be defined for tags not deeper than level 1 in an expression. In
> this post:
>
> http://stackoverflow.com/questions/5886066/overloading-seta-b-a-b/588...
>
> I demonstrated a method, which I also don't like, but which is at least
> safer because it tests for a
> specific data type. Generally, the still safer way is to create your o=
wn
> data type and introduce your own
> assignment operator(s) - in this way, you don't have to modify Set at all=
.
>
> Regards,
> Leonid
Oleksandr Rasputinov
I fully agree with all of your comments and of course you are right to
point out that modifying core functions such as Set is a bad idea in
general due to the likelihood of unintended consequences. Of course,
the performance issue introduced for setting parts of OwnValues can
be repaired to some extent:
indexedQ = Function[sym,
Symbol =!= Head@Unevaluated[sym] =!= Unevaluated[sym],
HoldFirst
];
Unprotect[Set];
Set[(sym_?indexedQ)[[part_]], val_] := (
sym = ReplacePart[sym, part -> val];
sym[[part]]
);
Protect[Set];
which restores the O(1) behaviour (albeit at the cost of an increased
constant factor). But even so, I will join with you in stressing that this
general approach is dangerous and should be attempted only with
extreme caution, if at all.
Best,
O. R.
On Jun 25, 10:30 am, Leonid Shifrin <lsh...@gmail.com> wrote:
> Hi Fabrice,
>
> My guess is that the reasons are efficiency and immutability of expres=
sions
> in Mtahematica. By restricting Set
> to variables or array elements, one can connect it directly to the intern=
al
> memory layout for that element. Note
> that all proposed solutions are based on ReplacePart, which copies the
> entire list to modify a single element.
> OTOH, constructs like indexed variables are serviced by hash-tables
> internally. Even if the value stored
> in the hash table is a pointer to some array, that array will likely be
> immutable. Whether it would be a
> good idea to make it mutable I don't know, but the general ideology is th=
at
> expressions are immutable
> in Mathematica. In other words, I don't think that the case of indexed
> variables was overlooked when
> designing Set - my guess is that the restriction to symbols is quite
> intentional.
>
> Regarding the "fix", I'd consider this not a fix but a recipe for disa=
ster,
> because modifying Set in such
> a fashion is dangerous, not to mention that it will often be completel=
y
> unacceptable performance-wise. Let us say
> you store a large array in a symbol and use a For loop with array indexin=
g.
> Suddenly all your array element assignments
> become O(n) time, where n is the list size, or even worse for higher
> dimensional tensors, not to mention increased
> memory consumption. Here is an illustration:
>
> In[18]:= a = Range[10000];
>
> In[19]:= Unprotect[Set];
> Set[sym_[[part_]], val_] := sym = ReplacePart[sym, part -> val];
> Protect[Set];
>
> In[27]:= Do[a[[i]] = a[[i]]^2, {i, Length[a]}] // Timing
>
> Out[27]= {1.156, Null}
>
> In[28]:= Unprotect[Set];
> Clear[Set];
> Protect[Set];
>
> In[31]:= Do[a[[i]] = a[[i]]^2, {i, Length[a]}] // Timing
>
> Out[31]= {0.047, Null}
>
> You may do this for yourself if you wish (although I'd avoid even that=
),
> but I'd strongly
> advise against such modifications in production code or any code someone
> else will be using. Set is one of the
> most fundamental operations, and its modifications may have very far-
> reaching and hard to predict consequences.
> It may, for example, corrupt the system in very subtle ways, which you ma=
y
> never be able to identify.
>
> Unfortunately, the standard mechanism of "soft" system symbols overloadin=
g
> via UpValues does not
> quite work with Set (or at least I don't know of a way to make it work)
> because Set holds its argument and UpValues
> can only be defined for tags not deeper than level 1 in an expression. In
> this post:
>
> http://stackoverflow.com/questions/5886066/overloading-seta-b-a-b/588...
>
> I demonstrated a method, which I also don't like, but which is at least
> safer because it tests for a
> specific data type. Generally, the still safer way is to create your o=
wn
> data type and introduce your own
> assignment operator(s) - in this way, you don't have to modify Set at all=
.
>
> Regards,
> Leonid
Leonid Shifrin
On Sun, Jun 26, 2011 at 2:26 PM, Oleksandr Rasputinov <
oleksandr_...@hmamail.com> wrote:
> Leonid,
>
> I fully agree with all of your comments and of course you are right to
> point out that modifying core functions such as Set is a bad idea in
> general due to the likelihood of unintended consequences. Of course,
> the performance issue introduced for setting parts of OwnValues can
> be repaired to some extent:
>
> indexedQ = Function[sym,
> Symbol =!= Head@Unevaluated[sym] =!= Unevaluated[sym],
> HoldFirst
> ];
>
> Unprotect[Set];
> Set[(sym_?indexedQ)[[part_]], val_] := (
> sym = ReplacePart[sym, part -> val];
> sym[[part]]
> );
> Protect[Set];
>
> which restores the O(1) behaviour (albeit at the cost of an increased
> constant factor). But even so, I will join with you in stressing that this
> general approach is dangerous and should be attempted only with
> extreme caution, if at all.
>
While your latest fix seems to work (except that it does not solve the O(n)
problem
for part assignments of arrays stored in indexed variables, but that at
least is not possible
to do without this code anyway, so no existing code can be broken. You also
could have included
the multi-index parts), I would not use it anyway.
In Mathematica, Hold-attributes give one the ability to make custom scoping
constructs and
assignment operators, a luxury that few languages offer. Overloading Set
(particularly by adding
DownValues to Set) therefore is, in the vast majority of cases, not a
necessity but just a
notational convenience, which one should weight against completely
unpredictable consequences of
redefing one of the most fundamental operations. And it does not matter how
carefully you use it, since there is a lot of top-level code in Mathematica itself,
and you will
never be able to guarantee that that code won't break. So the conclusion
should be
clear - create your own data type and assignment operators if you need. Too
bad that
Mathematica's grammar is fixed and currently no tools like preprocessor or
programmable
reader exist for it to make the code look prettier, but I think one can live
with that.
I must admit that I am myself guilty, having overloaded Set and SetDelayed a
few times, like e.g. here:
http://groups.google.com/group/comp.soft-sys.math.mathematica/browse_thread/thread/c000439b48751078
but I did it for very special purposes when I needed to "spy" on defintions
given when loading
packages, or modify them on the fly, and these were mostly for functions, so
more like a one-time
affair. And I am not fond of those hacks either.
Besides, Part assignment is anyway special and already overloaded in
Mathematica,
since for a generic head h which does not have Hold* Attributes, in an
assignment like
h[x] = value
both the head h, and value x, *are*evaluated, while rules for h[ _ ] are not
applied
(the fact that is not fully reflected in the documentation, which states
this for <x> but not for <h>).
But clearly, this is not what is happening in the case of Part, since Part
is not HoldFirst and
therefore in expr[[part]] = value, expr should normally have been
evaluated (and then would of
course no longer represent the L - value). So, by overloading Part
assignment, you not only
overload Set, but overload the already overloaded and not completely
specified part of Set definitions.
As an alternative to a custom part assignment operator, one can use a custom
wrapper for a part
assignment, at the expense of a slight notational inconvenience. For
example, like this (using your
indexedQ - which, to be pedantic, better be HoldAllComplete rather than
HoldFirst):
ClearAll[part];
part /: Set[part[sym_?indexedQ, part__], value_] :=
(
sym = ReplacePart[sym, part -> value];
sym[[part]]
);
part /: Set[part[sym_, part_], value_] := sym[[part]] = value;
part[expr_, part__] := Part[expr, part];
Since Set does not apply the rules for the head of its l.h.s., the
DownValues for
<part> won't fire before UpValues when we use <part> inside Set. Example of
use:
In[111]:= Clear[f];
f[1] = Range[10];
part[f[1], 3]
Out[113]= 3
In[116]:= part[f[1], 3] = 10;
part[f[1], 3]
Out[117]= 10
This immediately removes the main problem, since Set is now "softly"
overloaded via UpValues.
One can get fancier and then make a custom "scoping construct", which would
lexically or dynamically
change Part to part in a piece of code. Inside it, one can then use the
exact same syntax as before,
without any changes. Here is the lexical "scoping construct", for example:
ClearAll[withCustomPartLex];
SetAttributes[withCustomPartLex, HoldAll];
withCustomPartLex[code_] := With[{Part = part}, code];
Example of use:
withCustomPartLex[
Clear[f];
f[1] = Range[10];
f[1][[3]] = 10;
f[1]]
{1, 2, 10, 4, 5, 6, 7, 8, 9, 10}
And all is safe. The O(n) problem for assignments to parts of indexed
variables still not solved, since the same ReplacePart-based implementation
is used, but this is a separate topic. One way to solve it is to use the
temporary symbols generated during the first assignment to an indexed
variable, to store the data. One will have to deallocate those however.
Anyways, my main message is that even within what is available at the
top-level and already parsed Mathematica code, Mathematica metaprogramming
facilities are rich enough that one may, in most cases (including this one
IMO), satisfy one's taste in terms of both syntax and semantics, without
playing games with crucial and heavily used system functions.
Regards,
Leonid
Oleksandr Rasputinov
On Wed, 29 Jun 2011 22:31:47 +0100, Leonid Shifrin <lsh...@gmail.com>
wrote:
> Oleksandr,
>
> While your latest fix seems to work (except that it does not solve the
> O(n) problem
> for part assignments of arrays stored in indexed variables, but that at
> least is not possible
> to do without this code anyway, so no existing code can be broken. You
> also could have included
> the multi-index parts), I would not use it anyway.
>
> In Mathematica, Hold-attributes give one the ability to make custom
> scoping constructs and
> assignment operators, a luxury that few languages offer. Overloading Set
> (particularly by adding
> DownValues to Set) therefore is, in the vast majority of cases, not a
> necessity but just a
> notational convenience, which one should weight against completely
> unpredictable consequences of
> redefing one of the most fundamental operations. And it does not matter
> how carefully you use it, since there is a lot of top-level code in
> Mathematica itself, and you will
> never be able to guarantee that that code won't break. So the conclusion
> should be
> clear - create your own data type and assignment operators if you need.
Just to mention one other possibility, whereby overloading Set using
DownValues can be made safe by having the definitions apply only to
certain symbols using ideas from object-oriented programming. This
approach might even provide greater safety than the method you outline
below because it is symbol- (rather than scope-)specific. Of course
this is not to detract from your suggestions, which constitute an
excellent example of good practice in Mathematica.
Using indexedQ from before (which certainly should also be
SequenceHold, although HoldAllComplete seems perhaps excessive):
indexedSymbol /: new[indexedSymbol] :=
Module[{indexedSymbol},
(* Initialization *)
Unprotect[Set];
Set[sym_indexedSymbol?indexedQ[[part_]], val_] := (
sym = ReplacePart[sym, part -> val];
sym[[part]]
);
Protect[Set];
(* Destructor *)
indexedSymbol /: Clear[pre___, indexedSymbol, seq___] := (
Unprotect[Set];
Set[sym_indexedSymbol?indexedQ[[part_]], val_] =.;
Protect[Set];
indexedSymbol /: Clear[$pre___, indexedSymbol, $seq___] =.;
Clear[pre, indexedSymbol, seq]
);
indexedSymbol
];
Usage is then:
In[3] :=
f = new[indexedSymbol];
With[{f = f},
f[1] = Range[5];
f[1][[1]] = 10;
f[1]
]
Out[3] =
{10, 2, 3, 4, 5}
while behaviour for all other symbols remains unchanged. The
indexedSymbol object(s), when no longer required, are deleted using:
Clear[Evaluate[f], f];
At this point however we have a construct that is a better programming
exercise than it is a useful solution, given that it is more
complicated than using ReplacePart directly.
Best,
O. R.