replace values in a cell with nan
Andrew Stevens
I am trying to figure out a way to replace values in a cell array with NaN. I can do it with a loop, but I have somehow got myself into the CELLFUN habit and don't much like the look of the loop:
%generate some data
data=cellfun(@(x)(rand(x,1)),...
num2cell(ceil(rand(5,1)*100)),'un',0);
data =
[26x1 double]
[42x1 double]
[34x1 double]
[30x1 double]
[91x1 double]
%now if I want to replace values greater than 0.5 in each cell I could:
for i=1:length(data)
data{i}(data{i}>0.5)=NaN;
end
Is there a way to do it using CELLFUN? My initial attempt looked something like this:
data=cellfun(@(x)(x(x>0.5)=NaN),data,'un',0);
but that syntax is not valid. Any ideas?
Andrew
Bastian
I think this should work. It is a vectorized form of your loop:
data(data > 0.5) = NaN;
Bastian
Bastian
sorry, brain malfunction :) that obviously will not work!
Oleg Komarov
cellfun doesn't support explicit assignment (=).
cellfun(@(x) accumarray(find(x > 0.5),x(x > 0.5),[],[],NaN) , data,'un',0);
This one works only if any values of ALL cell meets the condition > 0.5.
If somebody finds another way...else you can convert to double with cell2mat and riconvert back to cell.
Oleg
Oleg Komarov
Typo, reverse the condition in the cellfun...still limitations remain, not to talk about performance...I would go for a loop...
Oleg
Walter Roberson
> "Andrew Stevens" <aste...@JUNKusgs.gov> wrote in message
> <hkvlbi$14j$1...@fred.mathworks.com>...
>> Hi all,
>>
>> I am trying to figure out a way to replace values in a cell array with
>> NaN. I can do it with a loop, but I have somehow got myself into the
>> CELLFUN habit and don't much like the look of the loop:
>>
>> %generate some data
>> data=cellfun(@(x)(rand(x,1)),...
>> num2cell(ceil(rand(5,1)*100)),'un',0);
>>
>> data =
>> [26x1 double]
>> [42x1 double]
>> [34x1 double]
>> [30x1 double]
>> [91x1 double]
>>
>> %now if I want to replace values greater than 0.5 in each cell I could:
>>
>> for i=1:length(data)
>> data{i}(data{i}>0.5)=NaN;
>> end
>>
>> Is there a way to do it using CELLFUN?
data = cellfun(@(x) x + (sign(inf * (x <= 0.5)) - 1), data);
Oleg Komarov
data = cellfun(@(x) x + (sign(inf * (x <= 0.5)) - 1), data,'un',0);
Just missing the 'un',0.
Oleg
Andrew Stevens
Walter,
That works perfectly. Now if only I could figure it out!
Thanks, Andrew
Nathan
> Walter Roberson <rober...@hushmail.com> wrote in message <hlhis4$jt...@canopus.cc.umanitoba.ca>...
> > Oleg Komarov wrote:
> > > "Andrew Stevens" <astev...@JUNKusgs.gov> wrote in message
> > > <hkvlbi$14...@fred.mathworks.com>...
> > >> Hi all,
>
> > >> I am trying to figure out a way to replace values in a cell array with
> > >> NaN. I can do it with a loop, but I have somehow got myself into the
> > >> CELLFUN habit and don't much like the look of the loop:
>
> > >> %generate some data
> > >> data=cellfun(@(x)(rand(x,1)),...
> > >> num2cell(ceil(rand(5,1)*100)),'un',0);
>
> > >> data =
> > >> [26x1 double]
> > >> [42x1 double]
> > >> [34x1 double]
> > >> [30x1 double]
> > >> [91x1 double]
>
> > >> %now if I want to replace values greater than 0.5 in each cell I could:
>
> > >> for i=1:length(data)
> > >> data{i}(data{i}>0.5)=NaN;
> > >> end
>
> > >> Is there a way to do it using CELLFUN?
>
> > data = cellfun(@(x) x + (sign(inf * (x <= 0.5)) - 1), data);
>
> Walter,
> That works perfectly. Now if only I could figure it out!
>
> Thanks, Andrew
The logic behind the algorithm seems to be as follows:
If x is larger than .5, the logic returns 0. 0 * inf = NaN. The sign
of NaN returns NaN. NaN - 1 = NaN. Therefore, NaN replaces the data.
If x is smaller than or equal to .5, the logic returns 1. 1 * inf =
inf. The sign of NaN returns 1. 1 - 1 = 0, therefore no change occurs
to the data.
I hope that helps.
-Nathan
Walter Roberson
> The logic behind the algorithm seems to be as follows:
> If x is larger than .5, the logic returns 0. 0 * inf = NaN. The sign
> of NaN returns NaN. NaN - 1 = NaN. Therefore, NaN replaces the data.
> If x is smaller than or equal to .5, the logic returns 1. 1 * inf =
> inf. The sign of NaN returns 1. 1 - 1 = 0, therefore no change occurs
> to the data.
Exactly, Nathan. It was just a matter of fishing through my memory and
doing a couple of tests until I found something an asymmetric operation
that produced NaN under some conditions and a numeric value under other
conditions. It helps that I've been doing a lot of stress-testing of
Maple lately, so I knew the kinds of corner-cases to look for.
Nathan
Ah. One small word switch: "The sign of NaN returns 1." to "The sign
of Inf returns 1."
Brilliant algorithm, by the way. It takes a lot of thinking to come up
with seemingly simple tasks like this.
-Nathan
Yair Altman
> data = cellfun(@(x) x + (sign(inf * (x <= 0.5)) - 1), data);
A minor improvement to Walt's great answer:
cellfun(@(x)x*sign(inf*(x<=.5)), data);
i.e., there is no need for -1 if we use x* instead of x+
Now, if Matlab only had a much-requested built-in IFF function (if x then y else z) we could have simplified this further to the much more readable:
cellfun(@(x) iff(x>.5,NaN,x), data);
of course, you can always code your own personal iff function for this:
function result = iff(flag, posResult, result)
if flag, result=posResult; end
end % iff
Yair Altman
http://UndocumentedMatlab.com
Jos (10584)
Here is a somewhat simpler alternative for the function:
@(x) x + 0 ./ ~(x>0.5)
hth
Jos
Steven Lord
"Yair Altman" <altma...@gmailDEL.comDEL> wrote in message
news:hliu3r$khr$1...@fred.mathworks.com...
... and this function would NOT do what the user wanted. Remember that when
you call IF with a nonscalar condition (as would be the case using the data
from the OP's example and your CELLFUN call) that ALL the elements of that
condition must be true in order for the condition to be considered true.
Therefore, to make it more explicit, your IFF function would actually be
equivalent to:
function result = iff(flag, posResult, result)
if all(flag)
result=posResult;
end
end % iff
So for the "much-requested built-in IFF function", would you want it to
demonstrate this behavior or to operate element-by-element, and more
importantly how many people do you believe would expect it to work the other
way? [I'm genuinely interested in your answers to the two parts of this
question.]
--
Steve Lord
sl...@mathworks.com
comp.soft-sys.matlab (CSSM) FAQ: http://matlabwiki.mathworks.com/MATLAB_FAQ
Jan Simon
> I am trying to figure out a way to replace values in a cell array with NaN. I can do it with a loop, but I have somehow got myself into the CELLFUN habit and don't much like the look of the loop:
After some nice solutions have been found, could you be so kind and compare the execution speed with a simple loop?
for iC = 1:numel(C)
aC = C{iC};
aC(aC >= 0.5) = NaN;
C{iC} = aC;
end
Although you don't "like" it, it could be more efficient.
Jan
PS. I do love the "comefrom" command even more than "goto" - but I'm not fond of the debugging hell these flow control methods yield to...
Matt Fig
>
> for iC = 1:numel(C)
> aC = C{iC};
> aC(aC >= 0.5) = NaN;
> C{iC} = aC;
> end
>
> Although you don't "like" it, it could be more efficient.
Or even simpler:
for ii = 1:numel(C)
C{ii}(C{ii}(:)>.5) = nan;
end
Oleg Komarov
> demonstrate this behavior or to operate element-by-element, and more
> importantly how many people do you believe would expect it to work the other
> way? [I'm genuinely interested in your answers to the two parts of this
> question.]
>
> --
> Steve Lord
> sl...@mathworks.com
> comp.soft-sys.matlab (CSSM) FAQ: http://matlabwiki.mathworks.com/MATLAB_FAQ
Not sure i understod your question but considering
cellfun(@(x) iff(x>.5,NaN,x), data);
I would expect
for ii = numel(data)
IDX = data{1} > .5
data{1}(IDX) = NaN
data{1}(~IDX) = x % (which is equal to itself in this specific case)
end
I wouldn't want
if all(data{1} > .5)
data{1}(:) = NaN; % neither data{1} = NaN;
else
data{1}(:) = x;
end
Oleg
Andrew Stevens
> Dear Andrew!
>
> > I am trying to figure out a way to replace values in a cell array with NaN. I can do it with a loop, but I have somehow got myself into the CELLFUN habit and don't much like the look of the loop:
>
> After some nice solutions have been found, could you be so kind and compare the execution speed with a simple loop?
>
> for iC = 1:numel(C)
> aC = C{iC};
> aC(aC >= 0.5) = NaN;
> C{iC} = aC;
> end
>
> Although you don't "like" it, it could be more efficient.
>
> Jan
Jan,
Of course you and Matt are right about the execution speed. For my application, I am not worried about speed.
In many cases for me, using cellfun over a loop is more compact, but that is not the case in this instance (Matt's loop has less characters than Walter's solution). So is there a reason to use the cellfun method for this problem (besides really confusing those you share your code with)? Not sure, but there are cases where Walters method will be slightly more compact:
%generate a structure containing cells
names={'andrew','bastian','walter',...
'oleg','nathan','yair',...
'jos','steven','jan','matt'};
data=cellfun(@(x,y)(rand(50,1)),...
num2cell(1:length(names)),'un',0)';
dstr=cell2struct(data,names);
%replace using walters method
d2=structfun(@(x)(x + (sign(inf * (x <= 0.5)) - 1)),...
dstr,'un',0);
%loop
d2=dstr;
fields=fieldnames(dstr);
for i = 1:length(fields);
d2.(fields{i})(d2.(fields{i})<=0.5) = NaN;
end
Anyway, thanks for all the input!
Andrew
Walter Roberson
> Here is a somewhat simpler alternative for the function:
>
> @(x) x + 0 ./ ~(x>0.5)
That's a nice improvement, Jos!!
It leads to a simplification by one matrix operation:
@(x) x + 0 ./ (x <= 0.5)
Steven Lord
Let me simplify the question by removing the extraneous CELLFUN. What would
your expectation be for the contents of y after these lines of code were
executed?
clear y
x = 0:0.1:1;
y = iff(x > 0.5, NaN, x)
Jan Simon
> > for iC = 1:numel(C)
> > aC = C{iC};
> > aC(aC >= 0.5) = NaN;
> > C{iC} = aC;
> > end
> for ii = 1:numel(C)
> C{ii}(C{ii}(:)>.5) = nan;
> end
Which one is faster on your machine?
I assume the aC method creates a deep copy, while C{ii){C{ii}(:))>.5) is not just a damn tricky smiley but used just shared data copies?
Jan
Oleg Komarov
0.1 0.2 0.3 0.4 0.5 NaN NaN NaN NaN NaN.
The other behavior can be described as follows:
iff(all(x > 0.5), NaN, x)
then if x = .6: .1 :1
x = NaN NaN NaN NaN NaN
Oleg
Yair Altman
cellfun(@(x) iff(x>.5,NaN,x), data)
Of course, data's cell values may themselves be non-scalars, in which case my simple IFF implementation would need to be more complex, but as far as I understood the OP's question, this was not the case, and I just wanted to show a basic solution template.
Again - I fully agree that a built-in IFF would need to handle cells, arrays, structs, objects and other complex types. But even a very basic implementation would be an improvement, because inline conditional statements are currently unavailable in CELLFUN/ARRAYFUN and their kin.
Yair
Matt Fig
Sorry Jan, I didn't see your reply.
On my machine the C method is consistently 20% faster than the aC and about 6 times faster than cellfun.