Hi Veronica,
Yes, you need to process the Ctrax output using PrepareJAABAData. Since you are running JAABA from source using MATLAB, you don't need to run the compiled PrepareJAABAData program. You can find PrepareJAABAData function in the perframe directory of the code (same location as StartJAABA.m) and you can run it from MATLAB command line. That way you won't have to deal with MCR's and other installation requirements.
And when you start a project, you don't give the path or name of a particular movie or mat file. In JAABA, there is separate folder for each movie (or experiment), and within this folder the name of the movie is always the same, say movie.avi or trx.mat. More details on how to prepare the folders are here
http://jaaba.sourceforge.net/DataFormatting.html. When you create a project, you enter the common name (movie.avi and trx.mat) in the movie file and trx file field.
HTH,
Mayank