MB program runs slower as input table increases in size

[Five Thumbs]

I created a MB program that reduces the number of points in a data set by aggregating based on a user defined distance.  I have noticed that as the number of points in the input file increases, the slower the program becomes.  For example, if the input has 1350 points the program processes 15.6 points per second.  If the input has 8300 points, processing decreases to 8.7 points per second. Can anyone give me a clue to why this is happening?  I have noticed this previously and somewhat resolved it by saving or packing the tables involved.

I used to use Vertical Mapper to aggregate points but that is no longer an option as it is not available. It could aggregate the aforementioned data sets in seconds as apposed to minutes.

James Harvie 

[Uffe Kousgaard]

You have probably developed yourself a O(n^2) function :-)

https://en.wikipedia.org/wiki/Big_O_notation#Orders_of_common_functions

That means you search through your list of points and picks one of them.
Now there is one less element.
Search again, but now with a shorter list.
Keep doing until all points has been processed.

That generally gives you that kind of behaviour.

MapBasic isn't very efficient for this kind of work, so a slow performance easier shows.


Regards
Uffe Kousgaard

--
--
You received this message because you are subscribed to the
Google Groups "MapInfo-L" group.To post a message to this group, send
email to mapi...@googlegroups.com
To unsubscribe from this group, go to:
http://groups.google.com/group/mapinfo-l/subscribe?hl=en
For more options, information and links to MapInfo resources (searching
archives, feature requests, to visit our Wiki, visit the Welcome page at
http://groups.google.com/group/mapinfo-l?hl=en

---
You received this message because you are subscribed to the Google Groups "MapInfo-L" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapinfo-l+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mapinfo-l/acd94721-63e8-47b8-98b9-bebf044c9263n%40googlegroups.com.

[Five Thumbs]

You are basically correct.  I have gone through several versions, each one becoming more efficient but farther away from the most ideal methodology.  My current methodology is as follows.

 

1.Fetch the first record in the point table

2. Determine the coordinates of the MBR around this point which includes the aggregation distance

3. Select all points which fall inside of the MBR

4. Determine the geographic center of those points and any other statistic like the number of points aggregate or average value of a selected field

5. Insert a point into a new table using the geographic center, include any statistic information calculated

6. Delete all records in the selection so they do not get selected again

7. Fetch next record and do it all again

8.After every 1000 records the table is saved and packed, the program is forced to fetch the first record.

I know that MB is slow, but should it continually slows down over the time it is running?  This usually means to me that it is running out of memory, and this is why I save and pack the table which saves me about 10% of the processing time.  Could there be another reason? Is there a way to do this work outside of MapInfo?  I know Vertical Mapper did aggregation outside.

James Harvie