Problem in setting sample time in Simulink

[davide]

Hi all,
I'm trying to solve this problem from 10 days, but it's still there!
I have a model that gets image from the webcam of my laptop (a macbook) and in realtime tracks people. To get image from the webcam I use a java class and its methods are called in the model by using a 2-level m-file s-function. Then the matrix associated to the image is sent in output to the model.
The code of the s-function is this:

function RealTime_video_from_webcam(block)
setup(block);
%end video_from_webcam

function setup(block)
% Register number of ports
block.NumInputPorts = 0;
block.NumOutputPorts = 1;
% Set output port properties
block.OutputPort(1).Dimensions = [640 640];
block.OutputPort(1).DatatypeID = 3; %uint8
block.OutputPort(1).Complexity = 'Real';
block.OutputPort(1).SamplingMode = 'sample';
block.OutputPort(1).SampleTime = [0.2 0];
%register methods
block.RegBlockMethod('Start', @Start);
block.RegBlockMethod('Outputs', @Outputs);
block.RegBlockMethod('Terminate', @Terminate);
block.RegBlockMethod('SetOutputPortSampleTime',@SetOutputPortSampleTime);
%end setup

function Start(block)
iSight = iSightInit([640 640]);
set_param(block.BlockHandle,'UserData', iSight);
%end Start

function SetOutputPortSampleTime(block, port, time)
block.OutputPort(port).SampleTime = time;
%end SetOutputPortSampleTime

function Outputs(block)
iSight = get_param(block.BlockHandle,'UserData');
block.OutputPort(1).Data = iSightCapture(iSight,'grayImage');
%end Outputs

function Terminate(block)
iSight = get_param(block.BlockHandle,'UserData');
iSightClose(iSight);
%end Terminate

It works, but not as I want. The problem is that even if I've set the sample time of the output port to 0.2, it samples the port every 1/1.5 second, with a very low frame rate. I need at least 10 frame per second. Now I only have a slow motion video!

Please, help me!!!
I'm getting crazy... I've tried everything... I've read matlab and simulink documentation from the first to the last page...
I don't have more ideas to solve the problem.
Does anyone know a possible way to fix it?

Thanks a lot in advance

[Omur]

"davide " <david...@inwind.it> wrote in message <gqqm6r$2k0$1...@fred.mathworks.com>...

I don't have a solution, but I can define the problem: Simulink does not work in real-time. 0.2 in Simulink can take 1 millisecond, 5 seconds, or 2 hours in real life. From what you said it looks like it's taking much longer than 0.2 sec, which is entirely possible.

If it were faster, I could suggest somewhat straightforward ways to slow it down to real-time, but when it's slow, you need to dig deep into your model and find ways to optimize its performance.

I hope this gives you an understanding of the problem,

Omur

[davide]

"Omur" <omur...@promodxxxx.com.tr> wrote in message <gqqn8h$c0h$1...@fred.mathworks.com>...

Yes, you've understood the problem, I want that simulink samples the output every 0.2 or less second, but it does every 1 or more time.
The model is already fast because if I send to it an avi video using the From Multimedia File block, it works at the same speed of the video. This means that if the video has a frame rate of 15 fps, it does its work 15 times per second, without problem.

I can say that what I'm trying to do in my s-function is the From Multimedia File block work, but getting images in realtime from a webcam.

if I can't work in realtime, I have to change all the project!!!
Please don't tell me that this is the only solution...

[wi...@mathworks.com]

David,

As one of the posters already noted, setting sample time, will not do much for you.
The reason for the slowdown is most likely due to one of these issues:
* the Java code is too slow to produce fast enough frame rate
* the M level 2 sfunction may introduce overhead that's slowing you down

Either way, you'll need to speed up your capture time. Do you have your Java source triggering the
acquisition of the next frame while you give control to the algorithmic part in Simulink? That is,
are you running your acquisition on a separate thread with some sort of ring buffer? If not, that
could be a reason for a slowdown. When your source is asked for data, you should be pulling it
out of the software buffer, and not asking your comera to capture it at that momement. I hope
that this is making sense.

To verify our assumptions, you can temporarily move to Windows. There, you'll have access to
the Image Acquisition Toolbox which may have a driver for your webcam. That would force you
off your MAC, but once you verify that things are working as expected, you can come back to
debugging your Java source on the MAC.

Good luck,

Witek

[davide]

The java code is not slow. I've calculated the time needed using tic-toc functions and is about 0.1 second.
The method that returns to the s-function the image from the webcam is iSightCapture(iSight,'grayImage'); and doesn't trigger the acquisition of the next frame while the control is to the algorithmic part in Simulink. But I think this is not a problem because, as I said above, the time needed is 0.1 second, while the Output functions of the s-function is sampled every 1/1.5 second.

In all the tests I've done, I tried also to execute the method 5 times in a for loop to see if all the 5 images are sent out. The method is executed 5 times, but nothing change, only the last of the 5 images is sent to the model.
The real problem is that the Outputs function of the S-function is sampled once every 1/1.5 second.

Anyway, I can try to modify the java source to capture the next image while the simulink model is computing the previous one.

I will post eventual change in the result.

wi...@mathworks.com wrote in message <gqr7vm$ope$1...@fred.mathworks.com>...

[wi...@mathworks.com]

Hi,

In this case, I think that this would be an interesting test: don't call the Java code and output
a constant buffer from your M level 2 s-function. What's the frame rate in this case?

Witek

[davide]

Hi,
to try this test i've written only this line in the Outputs function: block.OutputPort(1).Data = ones(640,640,'uint8');
Using a Frame Rate Display block, the frame rate obtained is variable from 0.95 to 1,2 frame per second. Instead executing the Java code, the frame rate is from 0,6 to 0,8.

Thanks a lot for your help

wi...@mathworks.com wrote in message <gqt5cr$g9o$1...@fred.mathworks.com>...

[davide]

I've tried to get images from the webcam using the From Video Device block of the Image acquisition toolbox on Windows, but it's still the same!
The frame rate is even slower, around 0,6 fps.
I don't know what more to do...

"davide " <david...@inwind.it> wrote in message <gqt7ha$cfk$1...@fred.mathworks.com>...

[wi...@mathworks.com]

David,

Ah, good to know that you are experiencing something similar with the Video Acq block. Are you working
fully with uint8 in your model? If so, the uint8 is considered a fixed point, so all the subsequent
processing will be done in fixed-point. The fixed point computations are "emulated" during the
normal (i.e. non-accelerated) simulation. Try the following:
* on windows - simply turn on the accelerator mode
* you may get even a better boost by going to configuaration section-> optimizations -> accelerator
compiler optimizations : turn them on

If you want your simulation to be fast without the use of an accelerator, use single data type for
your source.

Let us know what happened,

Thanks,

Witek

[davide]

On Windows, using the video acq block, the system returns error if uint8 data type is used. To avoid this I turned it to single and it works. In this case the simulation is faster, like from 1.5 to 2 and more fps.
Then I've tried to turn on the accelerator mode turning on also the compiler optimizations and this error happens:
Problem creating Accelerator MEX file for model 'trackpeople'. Error returned is:
Error using ==> accelbuild_private at 26
Error using ==> genMakefileAndBuild at 1010
Error(s) encountered while building model "trackpeople".
I don't understand what does it mean.
I have to tell you that I'm using Windows XP Pro SP3 on a virtual machine over my MAC. The VM has 1GB of dedicated RAM and Windows seems to work very good, like when you use it on a normal computer without VM.
Do you think I can gain speed on a native Windows system?

On MAC system, if I use single data type, it doesn't work in the sense that I get black images. The image returned by the java class is of type uint8 and for this reason I cast it to single data type. If I print the content of the new single type image, it is not a black image, but a normal one. When, instead, this image is sent outside the s-function towards the moedel, it is a black one.
Using instead uint8 data type and the accelerator mode turned on, the frame rate is higher than the Windows system, in particular is aorund 4.5 fps!

Now, the last 2 things to try are the accelerator mode on Windows (it is necessary to solve the error above) and the use of single mode on MAC (it is necessary to solve the problem of the black image).

Thanks a lot for your help

Davide

wi...@mathworks.com wrote in message <gr2huu$as5$1...@fred.mathworks.com>...

[Shankar]

Hi David,

Can you try accelerator mode with just the From Video Device block and To Video Display block on the windows system. It seems like some other block might be causing the issue for accelerator mode. What other blocks do you have in your model?

Does it show any error description apart from just saying that Errors were encountered during accelerator mode?

Thanks
Shankar

"davide " <david...@inwind.it> wrote in message <gr7e0r$rtn$1...@fred.mathworks.com>...

[davide]

Trying the accelerator mode just with From Video Device and Video Viewer blocks I got these messages in an error window:
Problem creating Accelerator MEX file for model 'prova'. Error returned is:

Error using ==> accelbuild_private at 26
Error using ==> genMakefileAndBuild at 1010

Error(s) encountered while building model "prova"
To see the build output, use set_param('prova','AccelVerboseBuild','on').
I got these other on the Matlab window:
### Building the Accelerator target for model: prova
C:/PROGRA~1/MATLAB/R2007b/bin\mex.bat -c -win32 OPTIMFLAGS="-DNDEBUG" -f C:/PROGRA~1/MATLAB/R2007b/bin/win32/mexopts/lccopts.bat -I. -I.. -IC:/PROGRA~1/MATLAB/R2007b\simulink\include -IC:/PROGRA~1/MATLAB/R2007b\extern\include -IC:/PROGRA~1/MATLAB/R2007b\rtw\c\src -IC:/PROGRA~1/MATLAB/R2007b\rtw\c\libsrc -IC:/PROGRA~1/MATLAB/R2007b\rtw\c\src\ext_mode\common -IC:/PROGRA~1/MATLAB/R2007b\rtw\c\src\ext_mode\tcpip -IC:/PROGRA~1/MATLAB/R2007b\rtw\c\src\ext_mode\serial -IC:/PROGRA~1/MATLAB/R2007b\rtw\c\src\ext_mode\custom -IC:/DOCUME~1/ADMINI~1/MYDOCU~1/MATLAB/PROVA_~1 -IC:/DOCUME~1/ADMINI~1/MYDOCU~1/MATLAB -IC:/PROGRA~1/MATLAB/R2007b/rtw/c/libsrc -IC:/PROGRA~1/MATLAB/R2007b/toolbox/rtw/imaq/imaqblks/win32/videoinput -IC:/PROGRA~1/MATLAB/R2007b\sys\lcc\include prova_acc.c
Error prova_acc.c: C:/PROGRA~1/MATLAB/R2007b/toolbox/rtw/imaq/imaqblks/win32/videoinput\simaqruntime.h: 49 unrecognized declaration
Error prova_acc.c: C:/PROGRA~1/MATLAB/R2007b/toolbox/rtw/imaq/imaqblks/win32/videoinput\simaqruntime.h: 63 unrecognized declaration
Error prova_acc.c: C:/PROGRA~1/MATLAB/R2007b/toolbox/rtw/imaq/imaqblks/win32/videoinput\simaqruntime.h: 72 unrecognized declaration
Error prova_acc.c: C:/PROGRA~1/MATLAB/R2007b/toolbox/rtw/imaq/imaqblks/win32/videoinput\simaqruntime.h: 83 unrecognized declaration
Error prova_acc.c: C:/PROGRA~1/MATLAB/R2007b/toolbox/rtw/imaq/imaqblks/win32/videoinput\simaqruntime.h: 90 unrecognized declaration
Error prova_acc.c: C:/PROGRA~1/MATLAB/R2007b/toolbox/rtw/imaq/imaqblks/win32/videoinput\simaqruntime.h: 99 unrecognized declaration
6 errors, 0 warnings

C:\PROGRA~1\MATLAB\R2007B\BIN\MEX.PL: Error: Compile of 'prova_acc.c' failed.

gmake: *** [prova_acc.obj] Error 2

### Real-Time Workshop build procedure for model: 'prova' aborted due to an error.

In conclusion I get the same errors using just these 2 blocks as when I use all the intermediate blocks of the complete model.

Thanks

Davide

[Shankar]

Hi David,

The errors explained it. The From Video Device block runs the Simulink Accelerator mode but requires a C++ compiler. I guess you are currently using C compiler (lcc?) currently. Is it possible for you to switch and try a C++ compiler? Let us know.

Thanks
Shankar

"davide " <david...@inwind.it> wrote in message <gr7tc1$ort$1...@fred.mathworks.com>...

[davide]

Actually I'm not using any compiler on Windows system becuase I've just installed it on a virtual machine over my Mac to try the Image Acq Toolbox.
Which compiler do you advice me to install?
After the installation, do I have to add the path of the compiler to the path of Matlab?

Thanks

"Shankar " <ssubramaR...@mathworks.com> wrote in message <gr8pg1$ldu$1...@fred.mathworks.com>...

[wi...@mathworks.com]

David,

There is a compiler that you can download from Microsoft for personal use. It does not have a UI but
you should not care sine you don't neet it. Once you get it and install it on your virtual machine,
simply type "mex -setup" in ML to select it. This should resolve your accelerator issue in Windows.
Running on virtual machine will affect you, but theoretically, it should still work.

On your MAC, after you obtain your video frame in Java, you need to scale it besides casting it
to Single. For floating point types, there is an assumed range that defines what's white and what's
black. That range is 0 to 1. Therefore, before you cast your data to single, divide it by 255.
That should take care of your "black" image.

This is great progress. You are really close to having higher frame rates. Good luck,

Witek

[Lawri]

I just want to add something I've picked up in school with someone else doing pictuergrabbing from a webcam, he haid verry slow capture rate for pictures, but delving into the useage of the camera he found it actually had 2 modes, 1 for reasonable pictures, but verry slow readout, and 1 for fast pictures for use in video.

I would also recommand you use what I call an atomic buffer setup, make a second buffer for the image that MATLAB reads from, which is atomic in nature and fast to read, and have JAVA write to a diffrent buffer while capturing, and move the captured buffer to the atomic buffer by copying it into a new object and switching the refrences of the read buffer for MATLAB (I hope this is clear). I would recommand a seperated thread for this.
that should prevent any blocking calls there might be with the imige capture, speeding things up. it does mean that untill a new frame is completed, you will still see the old frame.

hope this helps

[davide]

Hi,

I've installed both a Borland C++ compiler and an OpenWatcom C++ compiler. I took the right version to install from this page http://www.mathworks.com/support/compilers/release2007b/
I've tried to build the accelerator mode but still there are errors. This is what I get in case of using the Borland compiler:

### Building the Accelerator target for model: prova

MAKE Version 5.2 Copyright (c) 1987, 2000 Borland
Compiling prova_acc.obj
Borland C++ 5.5.1 for Win32 Copyright (c) 1993, 2000 Borland
.\prova_acc.c:
Compiling rt_nonfinite.obj
Borland C++ 5.5.1 for Win32 Copyright (c) 1993, 2000 Borland
.\rt_nonfinite.c:
Linking prova_acc
C:\PROGRA~1\MATLAB\R2007b\bin\mex -win32 -f C:\DOCUME~1\ADMINI~1\APPLIC~1\MATHWO~1\MATLAB\R2007b\mexopts.bat OPTIMFLAGS="-O2 -DNDEBUG" -outdir .. prova_acc.obj rt_nonfinite.obj C:\PROGRA~1\MATLAB\R2007b\rtw\c\lib\win32\rtwlib_rtwsfcn_bc55.lib C:\PROGRA~1\MATLAB\R2007b\toolbox\imaq\imaqblks\lib\win32\simaqutil.lib C:\PROGRA~1\MATLAB\R2007b\extern\lib\win32\borland\libut.lib C:\PROGRA~1\MATLAB\R2007b\extern\lib\win32\borland\libmwmathutil.lib
Turbo Incremental Link 5.00 Copyright (c) 1997, 2000 Borland
Error: 'C:\PROGRAM FILES\MATLAB\R2007B\TOOLBOX\IMAQ\IMAQBLKS\LIB\WIN32\SIMAQUTIL.LIB' contains invalid OMF record, type 0x21 (possibly COFF)

C:\PROGRA~1\MATLAB\R2007B\BIN\MEX.PL: Error: Link of '..\prova_acc.mexw32' failed.

** error 2 ** deleting ..\prova_acc.mexw32
The make command returned an error of 1
'An_error_occurred_during_the_call_to_make' is not recognized as an internal or external command,
operable program or batch file.

### Real-Time Workshop build procedure for model: 'prova' aborted due to an error.

Instead, the following is the error returned by the Openwatcom compiler:

### Building the Accelerator target for model: prova

Open Watcom Make Version 1.3
Portions Copyright (c) 1988-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See http://www.openwatcom.org/ for details.
### Compiling prova_acc.c
C:\WATCOM\binnt\wcc386 -bd -3s -e25 -ei -fpi87 -zp8 -zq -fr= -ox -DNDEBUG -DMATLAB_MEX_FILE prova_acc.c
### Compiling rt_nonfinite.c
C:\WATCOM\binnt\wcc386 -bd -3s -e25 -ei -fpi87 -zp8 -zq -fr= -ox -DNDEBUG -DMATLAB_MEX_FILE rt_nonfinite.c
### Linking ...
C:\PROGRA~1\MATLAB\R2007b\bin\mex -win32 -O -f C:\DOCUME~1\ADMINI~1\APPLIC~1\MATHWO~1\MATLAB\R2007b\mexopts.bat -outdir .. prova_acc.obj rt_nonfinite.obj C:\PROGRA~1\MATLAB\R2007b\rtw\c\lib\win32\rtwlib_rtwsfcn_openwatc13.lib C:\PROGRA~1\MATLAB\R2007b\toolbox\imaq\imaqblks\lib\win32\simaqutil.lib C:\PROGRA~1\MATLAB\R2007b\extern\lib\win32\watcom\libut.lib C:\PROGRA~1\MATLAB\R2007b\extern\lib\win32\watcom\libmwmathutil.lib
Open Watcom Linker Version 1.3
Portions Copyright (c) 1985-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See http://www.openwatcom.org/ for details.
loading object files
searching libraries
Error! E2028: __imp_exIMAQAcquireData is an undefined reference
Error! E2028: __imp_exIMAQGetLastError is an undefined reference
creating a Windows NT dynamic link library
file prova_acc.obj(C:\Documents and Settings\Administrator\My Documents\MATLAB\prova_accel_rtw\prova_acc.c): undefined symbol __imp_exIMAQAcquireData
file prova_acc.obj(C:\Documents and Settings\Administrator\My Documents\MATLAB\prova_accel_rtw\prova_acc.c): undefined symbol __imp_exIMAQGetLastError

C:\PROGRA~1\MATLAB\R2007B\BIN\MEX.PL: Error: Link of '..\prova_acc.mexw32' failed.

Error(E42): Last command making (..\prova_acc.mexw32) returned a bad status
Error(E02): Make execution terminated
The make command returned an error of 2
'An_error_occurred_during_the_call_to_make' is not recognized as an internal or external command,
operable program or batch file.

### Real-Time Workshop build procedure for model: 'prova' aborted due to an error.

What does is happening now?

Thanks for your patience