waitbar for parfor loops

[William Dampier]

I was wondering if anyone has found a way to do progressbars
for the new parfor loops in R2007b and 2008a.

For normal for-loops the simple waitbar command works well.
waitbar(0)
for i=1:N
waitbar(i/N)
...do something
end

however it doesn't work with the parfor loops, since things
aren't done in order.

Has anybody made a function or found a work-around?

Thanks
Will

[Narfi]

William,

Currently, there is unfortunately no way of receiving
notification in the MATLAB client when an iteration is
completed on a worker.

Best,

Narfi
"William Dampier" <wal...@gmail.com> wrote in message
<fs0m5p$nkd$1...@fred.mathworks.com>...

[William Dampier]

You don't necessarily need something that communicates when
an iteration is done ... you just need something that can
tell a waitbar to update.

I've tried something along these lines so far, maybe it will
inspire someone with more knowledge of the callback systems.

N=100;
BEEN_DONE_VEC=false(N,1)
wait_handle=waitbar(0);
parfor (IND=1:N)
BEEN_DONE_VEC(IND)=true;
waitbar(wait_handle,mean(BEEN_DONE_VEC))
...do something
end

In this case it doesn't matter that the operations are not
done in order, since it just updates it with the fraction of
instances that are completed. The only problem I have is
figuring out how to pass an 'update' type call to waitbar.

"Narfi " <narfi.st...@mathworks.com> wrote in message
<fs0tg5$qdq$1...@fred.mathworks.com>...

[jochen Kamm]

> N=100;
> BEEN_DONE_VEC=false(N,1)
> wait_handle=waitbar(0);
> parfor (IND=1:N)
> BEEN_DONE_VEC(IND)=true;
> waitbar(wait_handle,mean(BEEN_DONE_VEC))
> ...do something
> end

I had a similar idea. Matlab did not accept it though, since it will regard the mean(...) command (I used sum(...)) as something inconsistent as it varies with the order of the parfor loop execution - of which all results of course should be independent.

I thought of something more basic then:

fprintf(repmat('*',1,N));
parfor ind = 1 : N
fprintf('\b')
...
end

if N is too large fractioning is be necessary. Its crude but it works.

fractioning in parts n, choose n appriopriatly:
M = numel(find(mod(1:N,n)==0));
fprintf(repmat('*',1,M));
parfor ind = 1 : N
if mod(ind,n)==0; fprintf('\b'); end
...
end

[Gil Georges]

the following works for me:

function testParFor()

[Daniel]

"Will Dampier" wrote in message <fs0m5p$nkd$1...@fred.mathworks.com>...

I have done some searching myself and the only work around I could find is wiriting to a file and reading that file. (messy and slow). You still cannot update the waitbar at each parfor iteration....BUT....you can do x number of iterations in a parfor loop and then update the waitbar (i.e. loop a parfor loop and update each time parfor finishes). For example do the following:

matlabpool;
matlabpoolsize = matlabpool('size');
% do a for loop that is big enough to do all necessary iterations
for n=1:matlabpoolsize:ceil(length(testLengths/matlabpoolsize))
parfor z = n:min(n+(matlabpoolsize-1),length(testLengths))
% iteration code here
% note that z will be your iteration
end
% update waitbar each "matlabpoolsize"th iteration
waitbar(n/ceil(length(testLengths/matlabpoolsize)),wb);
end
close(wb);
matlabpool('close');

[Edric M Ellis]

"Daniel " <ddalg...@gmail.com> writes:

>> I was wondering if anyone has found a way to do progressbars
>> for the new parfor loops in R2007b and 2008a.

> I have done some searching myself and the only work around I could
> find is wiriting to a file and reading that file.

I have a similar workaround, but using sockets so it's possibly a tiny
bit tidier. It's here:

http://www.mathworks.com/matlabcentral/fileexchange/24594-parfor-progress-monitor

Cheers,

Edric.

[Carl]

Hey Will,

I tried all kinds of things to get this to work and finally just reworked my parfor into a dispatch and gather operation using parfeval (sketched below). The gather operation runs on the desktop worker so it can update the waitbar just fine and it can increment as the jobs asynchronously complete. I've also found, for everything I've tried so far, that this is actually faster than the original parfor loop - it just takes a little extra code.

%%%%parfor method:
parfor i = 1:N
p(i) = myfunc(arg1,arg2,arg3);
end

%%%%parfeval method:
pst = gcp; %grab parallel pool structure
h = waitbar(0,'Progress'); %initialize waitbar
for i = 1:N %dispatch jobs
f(i) = parfeval(pst,@myfunc,1,arg1,arg2,arg3,...);
end
for i =1:N %gather completed jobs
[completedIdx,p(i)] = fetchNext(f);
waitbar(i/N,h);
end
close(h);


I hope that is of some use to you.

Cheers,
Carl

"Will Dampier" wrote in message <fs0m5p$nkd$1...@fred.mathworks.com>...