fscanfMat Error
David Sidlauskas
file into a matrix using fscanfMat(). The data file has 1688 columns and 296
rows. I get the following error message:
Error info buffer is too small.
I take this to mean that the array size I am trying to read in is to large
for Scilab. I would like to ask if this is true, what the limits are, and
is there any work-around.
Thanks you very much in advance for any help.
Dave Sidlauskas
matthias klingel
i also get error message
'Error Info buffer is too small (too many columns in your file ?)'
for big files. However, it implies that only the number of columns (not
that of lines) is limited.
To go around this, use
Strings = read ('File',-1,1,'(a)');
Numbers = Evstr(Strings);
Anyhow, this is quite slow (because of evstr).
If you want to fix it permanently, you have to increase the INFOSIZE
constant in file SCI/routines/sound/fileio.c and recompile scilab. This
hack was mentioned not too long ago in a posting of lydia v.d., but i
was not able to find it anymore.
Cheers
Matthias
Lydia van Dijk
Dave> I have just downloaded Scilab (Windows Version)and am
Dave> trying to read a data file into a matrix using
Dave> fscanfMat(). The data file has 1688 columns and 296
Dave> rows. I get the following error message:
Dave> Error info buffer is too small.
Dave> I take this to mean that the array size I am trying to
Dave> read in is to large for Scilab. I would like to ask if
Dave> this is true,
Yes and no. ;-)
Yes, it is to much for fscanfMat(). No, Scilab can handle
larger matrices.
Dave> what the limits are,
I am not sure what the limitation in your case is, but a lot of
Scilab's i/o routines can only handle lines with less than 1024
characters.
Dave> and is there any work-around.
There are plenty, yet it sucks to hit a bogus limit like this
one.
Matt> i also get error message for big files. However, it
Matt> implies that only the number of columns (not that of
Matt> lines) is limited. To go around this, use
Matt> Strings = read ('File',-1,1,'(a)');
Matt> Numbers = Evstr(Strings);
Matt> Anyhow, this is quite slow (because of evstr).
Yup, your workaround is pretty sick.
Matt> If you want to fix it permanently, you have to increase
Matt> the INFOSIZE constant in file SCI/routines/sound/fileio.c
Matt> and recompile scilab. This hack was mentioned not too
Matt> long ago in a posting of lydia v.d., but i was not able to
Matt> find it anymore.
... works, and leaves the bogosity level unchanged.
Now, for my suggestion (takes 3.8 secs on a P3/550 with a matrix
of the size you mentioned):
mat = read('my-matrix-file', -1, 1688);
This is ok as long as the number of columns is known (or trivial
to compute). I've several other workarounds in my mind ... in
case you don't like this one.
HTH
-lvd
Sent via Deja.com http://www.deja.com/
Before you buy.
David Sidlauskas
Thanks for your suggestions. mat = read('mat.file', -1, 1688) works fine
for my current problem, but not having to specify the number of columns
would be nice.
You seem to be the resident expert on this group, so I hope that you don't
mind if I ask for your opinion. I have a project coming up where I'll be
dealing with large matrices, of the order 4,000*4,000. In particular, I'll
be calculating all of the eigenvectors of these matrices. Reasonable
graphics for visualizing this data will be valuable. The question is, which
is more appropriate, Scilab or Matlab? This is a one-off project so price is
an issue. I just did a quick test by finding the eigenvectors of an 844*844
matrix. Scilab took 332 seconds and Matlab 43 seconds (400Mhz P3).
I'd appreciate any comments. Both of these programs are wonderful as
compared to programming from scratch in C.
Dave Sidlauskas
You seem to be the resident expert on Scilab, so I hope you don't mind if I
ask for your opinion.
"Lydia van Dijk" <lydia_v...@my-deja.com> wrote in message
news:90onfe$skm$1...@nnrp1.deja.com...
Lydia van Dijk
In article <nwUX5.885$d62....@bgtnsc04-news.ops.worldnet.att.net>,
"David Sidlauskas" <dsidl...@worldnet.att.net> wrote:
> Thanks for your suggestions. mat = read('mat.file', -1, 1688) works
> fine for my current problem, but not having to specify the number of
> columns would be nice.
If the number of columns is unknown, you can't help reading at least
the first line of your matrix file, count the columns and use this
number. BTW, this is what fscanfMat does (well -- at least in
principle).
> You seem to be the resident expert on this group, so I hope that you
> don't mind if I ask for your opinion. I have a project coming up
> where I'll be dealing with large matrices, of the order 4,000*4,000.
... that's about 120MB for each matrix. Let's hope your machines have
enough RAM!
With this matrix size, and the associated computational burden, it is
important to pick the most closely matching algorithm for you specific
problem (e.g. real/complex, symmetric/Hermitean/indefinite matrix,
all/selected eigenvalues/-vectors, etc.)
> In particular, I'll be calculating all of the eigenvectors of these
> matrices. Reasonable graphics for visualizing this data will be
> valuable. The question is, which is more appropriate, Scilab or
> Matlab? This is a one-off project so price is an issue.
In all honesty, I cannot answer your question, but I can give you some
clues.
- If you have lots of Matla|3 heritage code, sticking with the
Commerical Product or switching to Octave, will cause you the least
headaches.
- If you need tightly integrated graphics, where e.g. you can select
points in a graphics window and feed a mouse-click back into your
script, Scilab is hard to beat.
- If you need extreme performance (even from badly written scripts),
you should look at Tela.
Both, Octave, and Tela feature a "loosely bound" graphics back-end.
They use Gnuplot, and PlotMTV respectively. (I am told, Tela-1.30
supports an additional plot program, called SPPC, which produces
OpenGL windows, PostScript, PPM, and GIF files. So, there are two
back-ends for one numerics environment.)
All three, Octave, Scilab, and Tela, are build for low-level
extension, in case performance is critical. Octave and Tela are
extended with C++-modules, Scilab preferably with C or Fortran-77
(C++, Ada, F90, cooperate with Scilab too, if you put on the white hat
:-).
For a more detailed comparison of Scilab and Tela, visit
http://www.hammersmith-consulting.com
> I just did a quick test by finding the eigenvectors of an 844*844
> matrix. Scilab took 332 seconds and Matlab 43 seconds (400Mhz P3).
I cannot reproduce your timings. Boggle?! Here is what I have found.
(All times are wall-clock seconds.)
The "Commercial Product" version 5.3.0 on a PIII/550 (Katmai
stepping):
>> n=844;a=rand(n,n);tic;[d,v]=eig(a);toc
elapsed_time =
172.9300
Octave-2.1.31 out-of-the-box
octave:1> n=844;a=rand(n,n);tic();[d,v]=eig(a);toc()
ans = 207.56
Tela-1.30 linked against ATLAS-3.0.3 w/ATLAS-aware LAPACK-3.0 on the
same CPU (but different machine and OS version ;-):
>n=844;a=rand(n,n);tic();[d,v]=eig(a);toc()
169.64
Scilab-2.5.1 out-of-the-box:
-->n=844;a=rand(n,n);timer();[d,v]=bdiag(a);timer()
ans =
457.61
Based on an educated guess, I would predict, that a tweaked Scilab
should take about 165-185s for the same problem.
> I'd appreciate any comments. Both of these programs are wonderful as
> compared to programming from scratch in C.
Right, in your case starting from scratch in C would be CAS (= computer
aided suicide). :->
HTH
Lydia van Dijk
David Sidlauskas
WOW. Thanks for all of the info. Guess I'm going to have to look at Octave
and Tela as well. I really appreciate your comments.
Dave Sidlauskas
"Lydia van Dijk" <lydia_v...@my-deja.com> wrote in message
news:912bfh$5j$1...@nnrp1.deja.com...
Julian Stoev
|Lydia,
|
|WOW. Thanks for all of the info. Guess I'm going to have to look at Octave
|and Tela as well. I really appreciate your comments.
|
Lydia was correct, but please try also Python with appropriate libs
(NumPy for example).
http://www.python.org/topics/scicomp/numpy.html
http://numpy.sourceforge.net/
Computer Science Team, X-Division, Lawrence Livermore National
Laboratory must know what they are doing... ;-)
--JS