Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Structure help

6 views
Skip to first unread message

Jonathan

unread,
Mar 11, 2010, 7:43:04 PM3/11/10
to
G'day,

I have the following code...

A = [1,2,3,4,]; %%%% DATA %%%%
lon = [-123.412482300000, -123.349197000000, -123.387645500000,-123.499196100000];
lat = [48.1177411000000,48.1170438000000,48.1850616000000,48.2661300000000];
N = length(A);

[test2(1:N).Geometry] = deal('Point'); %%%%% Create Structure %%%%%
for j = 1:N;
test2(j).Lon = lon(j);
test2(j).Lat = lat(j);
test2(j).id = A(j);
end

Is there a way I can speed up the process of creating the structure? In this example there are only four matrices that are integrated into the structure. However, I would like to do this for a 10*1000000+ data file. Using the above code, it took about 30 mins to do the first 200,000 lines of data.

Thanks
Jon

Matt J

unread,
Mar 11, 2010, 10:00:22 PM3/11/10
to
"Jonathan" <jka...@yahoo.co.uk> wrote in message <hnc2mo$qp3$1...@fred.mathworks.com>...

> Is there a way I can speed up the process of creating the structure? In this example there are only four matrices that are integrated into the structure. However, I would like to do this for a 10*1000000+ data file. Using the above code, it took about 30 mins to do the first 200,000 lines of data.

=====================

It's a really bad idea to split the data into structure elements this way, particularly in these large amounts. You're killing all of the memory contiguity that makes working with the data easy. I recommend that you work with the data in their original arrays.

TideMan

unread,
Mar 11, 2010, 10:50:08 PM3/11/10
to
On Mar 12, 4:00 pm, "Matt J " <mattjacREM...@THISieee.spam> wrote:
> "Jonathan" <jkak...@yahoo.co.uk> wrote in message <hnc2mo$qp...@fred.mathworks.com>...

> > Is there a way I can speed up the process of creating the structure?  In this example there are only four matrices that are integrated into the structure. However, I would like to do this for a 10*1000000+ data file.  Using the above code, it took about 30 mins to do the first 200,000 lines of data.
>
> =====================
>
> It's a really bad idea to split the data into structure elements this way, particularly in these large amounts. You're killing all of the memory contiguity that makes working with the data easy. I recommend that you work with the data in their original arrays.

I agree with Matt.
What are you trying to achieve here?
The only time I would use a structure array in this way is if I had
many different geometries, all with different numbers of components.
For geometry 1 you would read in the data and do:
test2(1).lon=lon; % this inserts the whole lon vector
test2(1).lat=lat;
test2(1).id=A;
Then for geometry 2 you would read in the data and do:
test2(2).lon=lon;
test2(2).lat=lat;
test2(2).id=A;
etc
etc

Jonathan

unread,
Mar 12, 2010, 11:42:11 AM3/12/10
to
I appreciate both your responses and you are exactly write. However, it appears to be the only way I can see how to create a geostructure (basically a GIS structure file) that can be converted by 'shapewrite - Mapping tooldbox' as a GIS shape file. I was able to create a single structure for the complete dataset but when using the shapewrite toolbox, the program produces an error. Hence, the need to create seperate geometry structures, which seems silly. I have contacted Matlab directly to see if they have any advice, but no luck so far. Again thanks for the input.

Jon

TideMan <mul...@gmail.com> wrote in message <d25acd9a-c374-4115...@t17g2000prg.googlegroups.com>...

Jan Simon

unread,
Mar 12, 2010, 4:39:24 PM3/12/10
to
Dear Jonathan!

"The first 200,000 lines of data"? So you are reading from a file? This is not shown in the code snippet. Are you sure that the assigning to the struct is the bottleneck?

Jan

Jonathan

unread,
Mar 15, 2010, 12:20:43 PM3/15/10
to
The code is just an example. However, what we are trying to do is read in approximately 1.2 million nodes, that contain a variety of information that relate to modeling tsunami inundation. This is straight-forward, including clipping the data to a specific boundary. We were hoping to do most of the data processing in matlab, and ultimately to output the data as an ArcGIS shapefile. However, the latter is proving challenging since it appears that every point has to have its structure, with associated attributes, which means a lot of processing.

"Jan Simon" <matlab.T...@nMINUSsimon.de> wrote in message <hnecac$1lh$1...@fred.mathworks.com>...

Matt J

unread,
Mar 15, 2010, 1:12:15 PM3/15/10
to
"Jonathan" <jka...@yahoo.co.uk> wrote in message <hndqt3$q03$1...@fred.mathworks.com>...

> I appreciate both your responses and you are exactly write. However, it appears to be the only way I can see how to create a geostructure (basically a GIS structure file) that can be converted by 'shapewrite - Mapping tooldbox' as a GIS shape file. I was able to create a single structure for the complete dataset but when using the shapewrite toolbox, the program produces an error. Hence, the need to create seperate geometry structures, which seems silly. I have contacted Matlab directly to see if they have any advice, but no luck so far. Again thanks for the input.
=======================

I'm not too familiar with the Mapping Toolbox, but possibly it would be enough to fool shapewrite it into thinking that the syntax myStruct.lat(j)
is the same as myStruct(j).lat

That way you can construct a single struct-like object whose fields lat, lon, id are your original data arrays, but it could be indexed as myStruct(j).lat with the same effect as if it were structure array.

To do this you could define your own MATLAB data class with appropriate subsref and subsasgn methods. The classdef file down at the bottom of this post is the beginnings of such an implementation.


Example:

First, I'll cook up some dummy data and create an object test2

lat=1:10;
lon=lat*10;
id=1000:1005;
test2=myClass(lat,lon,id);

From the following, it may look to you like the variable test2 that I've created here is just a scalar struct

>> test2.lat,test2.lon,test2.id

ans =

1 2 3 4 5 6 7 8 9 10


ans =

10 20 30 40 50 60 70 80 90 100


ans =

1000 1001 1002 1003 1004 1005


But notice what else I can do


>> test2(3).lat, test2.lat(3)

ans =

3


ans =

3

>> test2(3).lon, test2.lon(3)

ans =

30


ans =

30

>> test2(3).id, test2.id(3)

ans =

1002


ans =

1002

%%%%%%To be put in file called myClass.m
classdef myClass

properties

mystruct;

end



methods

function obj=myClass(lat,lon,id)

mystruct.lat=lat;
mystruct.lon=lon;
mystruct.id=id;
obj.mystruct=mystruct;

end


function out=subsref(obj,S)

if ~ischar(S(1).subs)
S=S(end:-1:1);
end

out=subsref(obj.mystruct,S);

end

function obj=subsasgn(obj,S,rhs)

if ~ischar(S(1).subs)
S=S(end:-1:1);
end

obj.mystruct=subsasgn(obj.mystruct,S,rhs);

end

end

end

0 new messages