A common standard to share libraries between Matlab alternatives

[Chuong]

Hi Eugene,

Re: What we are working on...
"
...
Samit is finishing up new array class. The new arrays will simplify
writing matlab functions and improve performance.
...
"

Do you and Samit have any plan to develop a standard interface so that
most of FreeMat built-in functions (or its libraries) can be accessed
easily from a stand alone C program or even from Octave or other
Matlab alternatives?

I believe that all Matlab alternatives should establish a common
standard to share their libraries. If this is possible, developers can
save their time rewriting the same functions, and concentrate on
building a more complete Matlab alternative platform.

Chuong

[Chuong]

I mean a common standard that allows flexible access from different
software without sacrificing performance.

[EI]

Most of the executable code in Freemat implements language infrastructure and graphics shell. It would be hard to reuse that part. Most of the mathematical algorithms end up in matlab scripts and would be easy to share if octave and scilab used matlab syntax rather than their own dialect.

One thing would help is to cleanup freemat mex support to make it fully compatible. Then any executable extensions that are written for matlab could be easily recompiled for freemat.

Eugene

[demetrios...@googlemail.com]

> Most of the executable code in Freemat implements language infrastructure
> and graphics shell. It would be hard to reuse that part. Most of the
> mathematical algorithms end up in matlab scripts and would be easy to share
> if octave and scilab used matlab syntax rather than their own dialect.

+1 :).
This is the main reason a reuse from Octave was impossible :).
Regarding Matlab: well, I understand that they try to obfuscate as
much as possible to avoid "free clones" :).
On can observe as m code (from the distributio) become more and more
obfuscated (but not really faster) with each new version :).

> One thing would help is to cleanup freemat mex support to make it fully
> compatible. Then any executable extensions that are written for matlab could
> be easily recompiled for freemat.

This sounds interesting, but I think the main purpose would be to have
"script compatiblity" first, i.e. scripts written in matlab by users
to run without problems in FreeMat, so that the users to have an
"alternative for free" :).

Demetrios.

[Timothy Cyders]

   > One thing would help is to cleanup freemat mex support to make it fully

> compatible. Then any executable extensions that are written for matlab could
> be easily recompiled for freemat.

This sounds interesting, but I think the main purpose would be to have
"script compatiblity" first, i.e. scripts written in matlab by users
to run without problems in FreeMat, so that the users to have an
"alternative for free" :).

I agree, when I presented this past week, that was the biggest question - 'I've written  a lot of scripts and m-file programs in MATLAB, will I have to go rewrite and debug them all, or will they translate smoothly?' Especially for engineers, many times the programming is rather simple, so just making such changes as fprintf functionality (as was done in 3.6), etc. will have some significant impact on acceptance of FreeMat.

TJ

Demetrios.

[Chuong]

On Apr 19, 4:17 am, EI <gene...@gmail.com> wrote:
> Most of the executable code in Freemat implements language infrastructure
> and graphics shell. It would be hard to reuse that part. Most of the
> mathematical algorithms end up in matlab scripts and would be easy to share
> if octave and scilab used matlab syntax rather than their own dialect.
>

I see that FreeMat starts to grow to a stage that it has full language
infrastructure and new C codes will be mainly for built-in functions
that are too time-consuming to run if programmed in m files.

> One thing would help is to cleanup freemat mex support to make it fully
> compatible. Then any executable extensions that are written for matlab could
> be easily recompiled for freemat.
>

I also look forward to mex support. Though it seems that mex is more
suitable for user-defined functions, it's the only accepted standard
at the moment.

Chuong

[Chuong]

> > Most of the executable code in Freemat implements language infrastructure
> > and graphics shell. It would be hard to reuse that part. Most of the
> > mathematical algorithms end up in matlab scripts and would be easy to share
> > if octave and scilab used matlab syntax rather than their own dialect.
>
> +1 :).
> This is the main reason a reuse from Octave was impossible :).

Yes, although Octave also tries to run Matlab m files without
modification, it's internal m files can not run on Matlab. Porting
many of Octave m files are still possible and takes less time than
rewriting them.

> > One thing would help is to cleanup freemat mex support to make it fully
> > compatible. Then any executable extensions that are written for matlab could
> > be easily recompiled for freemat.
>
> This sounds interesting, but I think the main purpose would be to have
> "script compatiblity" first, i.e. scripts written in matlab by users
> to run without problems in FreeMat, so that the users to have an
> "alternative for free" :).

Making FreeMat codes run on Matlab seems much easier than the
opposite. But we decided to stay close to Matlab as much as possible,
didn't we? Probably we need to make a set of goals and development
guidelines for FreeMat. Sometimes I am reluctant to make some
modifications to FreeMat codes because I am not sure my codes will
cause trouble in the long term.

Chuong

[Chuong]

> I agree, when I presented this past week, that was the biggest question -
> 'I've written  a lot of scripts and m-file programs in MATLAB, will I have
> to go rewrite and debug them all, or will they translate smoothly?'
> Especially for engineers, many times the programming is rather simple, so
> just making such changes as fprintf functionality (as was done in 3.6), etc.
> will have some significant impact on acceptance of FreeMat.
>

Another idea to make FreeMat more accepted is to provide some hardware
support such as USB camera, RS232 serial connection, etc. Students
would love to have such features for their experiments. For sound
support, it seems FreeMat has different function naming from that in
Matlab.

Chuong

[Timothy Cyders]

Another idea to make FreeMat more accepted is to provide some hardware
support such as USB camera, RS232 serial connection, etc. Students
would love to have such features for their experiments. For sound
support, it seems FreeMat has different function naming from that in
Matlab.

Chuong

I totally agree. This is an area where an open-source solution can stand out from the commercial applications. I've used some simple scripting to get data from my GPS into a matrix in FreeMat, and it's been incredibly useful. To be able to read voltages, etc. would make FreeMat an instantly valuable tool for just about any engineer looking to run simple (or complex!) tests.

[Chuong]

Hi Timothy,

> I totally agree. This is an area where an open-source solution can stand out
> from the commercial applications. I've used some simple scripting to get
> data from my GPS into a matrix in FreeMat, and it's been incredibly useful.
> To be able to read voltages, etc. would make FreeMat an instantly valuable
> tool for just about any engineer looking to run simple (or complex!) tests.

Sound card can be used as a cheap analog to digital converter (ADC),
so you can use it to read voltage. I don't have any experience working
with USB interface to external hardwares such as GPS, ADC, etc. Do you
actually read real-time data from your GPS system? If so, could you
please share your experience so that we can add that ability to
FreeMat? We really need some hardware experts to extend FreeMat in
this direction. It also would be nice if we have some case studies for
demonstration.

Thanks,
Chuong

[Chuong]

> I see that FreeMat starts to grow to a stage that it has full language
> infrastructure and new C codes will be mainly for built-in functions
> that are too time-consuming to run if programmed in m files.

I am a bit wrong here as JIT will help speed up m codes. Ideally all
functions should be written in m codes as they would be more compact,
more portable and faster to develop and maintain. It would be nice if
JIT can compile optimized m codes (and save compiled codes like Matlab
compiler) and provide similar performance as written in C (by FreeMat
5.0 perhaps). The need to write functions in C will be greatly
reduced.

> Making FreeMat codes run on Matlab seems much easier than the
> opposite.

For user point of view, it is important to have codes written for
FreeMat run without modifications on Matlab. This is especially true
when user-defined functions initially written for FreeMat are then
used by a program which can only run on Matlab when it requires some
special functions not available on FreeMat. At the moment, codes using
FreeMat external interface (with import function) is not portable to
Matlab. I wonder if this feature can be wrapped into a independent
package with mex interface so that it can be added to Matlab or any
Matlab alternatives that support mex interface.

It would also be nice to have a variant of FreeMat external interface
which allows mixing C codes and m codes in the same file like
weave.inline for Python (http://www.scipy.org/PerformancePython). This
would ease development packaging efforts.

Chuong

[Chuong]

> I see that FreeMat starts to grow to a stage that it has full language
> infrastructure and new C codes will be mainly for built-in functions
> that are too time-consuming to run if programmed in m files.

[Timothy Cyders]

Actually, I'm working on getting a usable simple hardware implementation for reading AC and DC voltages into a mic port (the problem being that DC voltages/offsets are eliminated automatically by hardware, I'm getting around this by converting DC voltage to AC amplitude, then extracting data inside the program, but it's a work in progress). If you've got any ideas along those lines, I'm totally open. As far as getting GPS data into freemat, it was simple in linux with a garmin GPS. Connected to a serial port, you can use dgps or gpsbabel or any of a myriad of other scripts to pull of track data...my laptop is sans serial port, though, so I have a serial to usb converter, and simply query the usb port with the GPS connected and on:

cat /dev/ttyUSB0>output.data

note: sometimes this will require a chmod of /dev/ttyUSB0, as it is usually owned by root and restricted. You could also run as root. At any rate, this puts the data into a file, line by line as a series of numbers and characters according to a normalized scheme in terms of longitude, latitude, time, etc. I wrote a freemat script to decode this data using fopen, fgetline and str2num, using strncmp to examine the character values (e.g.:e vs. w). This reads the data into a matrix arranged into columns of decimal time, decimal longitude, decimal latitude, distance, velocity...whatever you like. I can upload the m file if you think it would be useful...it's pretty rudimentary, but it works for my purposes. For windows, one can load track data from a spreadsheet into a .csv file, then read it with csvread. This is what I did before I had my eeepc and a sudden urge to get data real-time.

Any suggestions on what would make scripts like this more useful?

TJ

[Chuong]

Hi Timothy,

Have you tried something like this within FreeMat?
fid = fopen("/dev/ttyUSB0", "r+")
fwrite(fid, your_data)
out = fread(fid)
I found this example from http://www.nabble.com/Re%3A-RS232-Serial-Comms-p7009507.html

Chuong

[Timothy Cyders]

I gave this a shot today, and it somewhat works. I had to make my own modifications, here's what I did:

first, at the terminal, you have to setup the USB for data querying at a given baud rate:

sudo stty /dev/ttyUSB0 ispeed 4800

then in freemat, simply:

fid = fopen('/dev/ttyUSB0','r+');
n = fgetline(fid);

You can run this as a loop with a wait(1) feature to make sure you don't catch up with the end of the file (the file continues to grow each second as the GPS adds a query line), or you could simply let it run with a wait command if you reach the end of the file (a simple if feof(fid) wait(1) statement ought to do). Note: Garmin Etrex GPS, text output mode. Every other line will be blank, so usually my code looks something like this:

for i = 1:1000
fid = fopen('/dev/ttyUSB0','r+'); % opens /dev/ttyUSB0 for data query
n = fgetline(fid); % gets blank line
m = fgetline(fid); % gets data line

n and m will be strings, with the following format (I've already written a script to decode this into useful data, I'll upload it when I get to my work computer tomorrow):


<<note: borrowed from http://www.zerofusion.ca/moike/cabling.html>>

Simple Text Output Format:

The simple text (ASCII) output contains time, position, and velocity data in
the fixed width fields (not delimited) defined in the following table:

    FIELD DESCRIPTION:      WIDTH:  NOTES:
    ----------------------- ------- ------------------------
    Sentence start          1       Always '@'
    ----------------------- ------- ------------------------
   /Year                    2       Last two digits of UTC year
  | ----------------------- ------- ------------------------
  | Month                   2       UTC month, "01".."12"
T | ----------------------- ------- ------------------------
i | Day                     2       UTC day of month, "01".."31"
m | ----------------------- ------- ------------------------
e | Hour                    2       UTC hour, "00".."23"
  | ----------------------- ------- ------------------------
  | Minute                  2       UTC minute, "00".."59"
  | ----------------------- ------- ------------------------
   \Second                  2       UTC second, "00".."59"
    ----------------------- ------- ------------------------
   /Latitude hemisphere     1       'N' or 'S'
  | ----------------------- ------- ------------------------
  | Latitude position       7       WGS84 ddmmmmm, with an implied
  |                                 decimal after the 4th digit
  | ----------------------- ------- ------------------------
  | Longitude hemishpere    1       'E' or 'W'
  | ----------------------- ------- ------------------------
  | Longitude position      8       WGS84 dddmmmmm with an implied
P |                                 decimal after the 5th digit
o | ----------------------- ------- ------------------------
s | Position status         1       'd' if current 2D differential GPS position
i |                                 'D' if current 3D differential GPS position
t |                                 'g' if current 2D GPS position
i |                                 'G' if current 3D GPS position
o |                                 'S' if simulated position
n |                                 '_' if invalid position
  | ----------------------- ------- ------------------------
  | Horizontal posn error   3       EPH in meters
  | ----------------------- ------- ------------------------
  | Altitude sign           1       '+' or '-'
  | ----------------------- ------- ------------------------
  | Altitude                5       Height above or below mean
   \                                sea level in meters
    ----------------------- ------- ------------------------
   /East/West velocity      1       'E' or 'W'
  |     direction
  | ----------------------- ------- ------------------------
  | East/West velocity      4       Meters per second in tenths,
  |     magnitude                   ("1234" = 123.4 m/s)
V | ----------------------- ------- ------------------------
e | North/South velocity    1       'N' or 'S'
l |     direction
o | ----------------------- ------- ------------------------
c | North/South velocity    4       Meters per second in tenths,
i |     magnitude                   ("1234" = 123.4 m/s)
t | ----------------------- ------- ------------------------
y | Vertical velocity       1       'U' (up) or 'D' (down)
  |     direction
  | ----------------------- ------- ------------------------
  | Vertical velocity       4       Meters per second in hundredths,
   \    magnitude                   ("1234" = 12.34 m/s)
    ----------------------- ------- ------------------------
    Sentence end            2       Carriage return, '0x0D', and
                                    line feed, '0x0A'
    ----------------------- ------- ------------------------

If a numeric value does not fill its entire field width, the field is padded
with leading '0's (eg. an altitude of 50 meters above MSL will be output as
"+00050").

Any or all of the data in the text sentence (except for the sentence start
and sentence end fields) may be replaced with underscores to indicate
invalid data.

[Timothy Cyders]

Oh, and by the way, I hope to have a simple voltage reading program that can use wavrecord() and wavread() interfacing with a rather simple circuit I'm in the process of building to form a pretty useful A/D converter. It seems to work so far, but I need to run some calibrations and accuracy/precision tests for validation.

TJ

[Chuong]

Very interesting, though I don't have a GPS to play with. It will be
also interesting to know the best dynamic range your ADC can achieve.

Demetrios, do you think FreeMat blog is a better place to put
Timothy's work as a fun and useful application of FreeMat? The blog
invitation sent to me has expired. Could you please send another?
Thanks.

By the way, I just came across Matwrap at http://lnc.usc.edu/~holt/matwrap/
. It utilizes extra array information to automatically create a
wrapper. For example, a C function written like this:

myfunc(const char *mat, unsigned m, unsigned n, char *mat_out)
{
...
}
//%input mat(m,n)
//%output mat_out(m,n)

will automatically get a wrapper so that it can be used in Matlab or
others like this :

mat_out = myfunc(mat)

This looks like the right solution to provide a simple standard
library interface for Matlab type software. We can follow the same
coding style to make a quick improvement to FreeMat external interface
to automatically create a suitable m wrapper around a compiled generic
C function. Thanks to import function, this can be done more easily
inside FreeMat. Still in this way, I can't see how C's printf and C+
+'s cout can be used to send messages to Command Window.

It looks like mex support in FreeMat is not very far from complete.
There are some functions remaining to be implemented. I have no idea
how to test it. Could anyone please point out the information about
how to test FreeMat's mex support and improve it?

Chuong

On Apr 29, 4:25 pm, "Timothy Cyders" <t.cyd...@gmail.com> wrote:
> Oh, and by the way, I hope to have a simple voltage reading program that can
> use wavrecord() and wavread() interfacing with a rather simple circuit I'm
> in the process of building to form a pretty useful A/D converter. It seems
> to work so far, but I need to run some calibrations and accuracy/precision
> tests for validation.
>
> TJ
>

> On Tue, Apr 29, 2008 at 2:23 AM, Timothy Cyders <t.cyd...@gmail.com> wrote:
> > I gave this a shot today, and it somewhat works. I had to make my own
> > modifications, here's what I did:
>
> > first, at the terminal, you have to setup the USB for data querying at a
> > given baud rate:
>
> > sudo stty /dev/ttyUSB0 ispeed 4800
>
> > then in freemat, simply:
>
> > fid = fopen('/dev/ttyUSB0','r+');
> > n = fgetline(fid);
>
> > You can run this as a loop with a wait(1) feature to make sure you don't
> > catch up with the end of the file (the file continues to grow each second as
> > the GPS adds a query line), or you could simply let it run with a wait
> > command if you reach the end of the file (a simple if feof(fid) wait(1)
> > statement ought to do). Note: Garmin Etrex GPS, text output mode. Every
> > other line will be blank, so usually my code looks something like this:
>
> > for i = 1:1000
> > fid = fopen('/dev/ttyUSB0','r+'); % opens /dev/ttyUSB0 for data query
> > n = fgetline(fid); % gets blank line
> > m = fgetline(fid); % gets data line
>
> > n and m will be strings, with the following format (I've already written a
> > script to decode this into useful data, I'll upload it when I get to my work
> > computer tomorrow):
>

> > <<note: borrowed fromhttp://www.zerofusion.ca/moike/cabling.html>>