Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Getting MATLAB to read large data sets

2,856 views
Skip to first unread message

Justin

unread,
Feb 10, 2009, 4:35:02 AM2/10/09
to
I was wondering if anyone knew how to get MATLAB to read large data sets. By large I mean 365 x 82 x122 (time/latitude/longitude). When I click on the data set MATLAB recognises that it is 'present' but I can't get access to the data to see what the values are - when I try and do this (by clicking on 'data' in the Workspace area) MATLAB says: 'Cannot display variables with more than 524288 elements.'

Is there any way that I can get MATLAB to read the data in the current format? I have tried exporting the data set into excel but it cannot 'read' the data either - it just comes up with symbols/letters...

If anyone knows a way round this I'd be very grateful.

Best Wishes,

Justin.

David

unread,
Feb 10, 2009, 5:47:01 AM2/10/09
to
"Justin " <J.Kr...@uea.ac.uk> wrote in message <gmrho5$9jj$1...@fred.mathworks.com>...

ok, so if you can click on it in the workspace then matlab has 'read' the data, so you don't have a problem with reading. you are having a problem with displaying the data... and you have found the limit of size of the array editor. so to 'see' the data you now have to use some other method to extract part of it, or plot it... say your data is in the variable A, you can just type A(1,2,3) and look at a single element. or something like A(:,1,1) to look at a vector from it. or even A(1,:,:) to look at a whole page of data. you may also want to look at surface plots and other 2d/3d plotting methods.

Justin

unread,
Feb 10, 2009, 7:08:02 AM2/10/09
to
Thanks for your useful comments David.

Justin.

"David" <da...@bigcompany.com> wrote in message <gmrlv5$a83$1...@fred.mathworks.com>...

Steven Lord

unread,
Feb 10, 2009, 10:39:52 AM2/10/09
to

"Justin " <J.Kr...@uea.ac.uk> wrote in message
news:gmrho5$9jj$1...@fred.mathworks.com...

>I was wondering if anyone knew how to get MATLAB to read large data sets.
>By large I mean 365 x 82 x122 (time/latitude/longitude). When I click on
>the data set MATLAB recognises that it is 'present' but I can't get access
>to the data to see what the values are - when I try and do this (by
>clicking on 'data' in the Workspace area) MATLAB says: 'Cannot display
>variables with more than 524288 elements.'

As the message indicates, the Array Editor can't display variables with more
than 524288 [=2^19] elements. If I remember correctly, that's for
performance reasons.

However, you can look at the data (or slices thereof) in the command window
by indexing into the variable, or you can extract slices that have fewer
than 2^19 elements and view those slices in the Array Editor. If your data
is stored in A, to extract all the latitudes and longitudes for a given
time, use:

t5 = A(5, :, :) % All lats and longs for the time corresponding to row 5.

--
Steve Lord
sl...@mathworks.com


0 new messages