Analysis of memory usage based on complexity/length of route and map

[Harry van der Wolf]

Hi all,

(If this mail layout with embedded table does not function in your mail client you can also read the attached pdf)

OsmAnd is a memory hungry program when it comes to route calculation to enable your navigation, and also for displaying the generated route.

About a week ago I wanted to calculate a longer route with OsmAnd and experienced constant crashes. Asking about this in a post turned out that the max distance was about 800 kilometers and I thought this was just a limit of Osmand (bummer!). A few days later I concluded that this had to be not OsmAnd itself but the limit on the amount of memory Osmand was allowed to use.

Having created many custom roms for many Archos based tablets until a year ago I decided to start playing with the heap memory on my phone.

Most phones and tablets still come with the heapsize memory parameter set to 32m, e.g. dalvik.vm.heapsize=32m (32m is 32 MegaByte). The heap size is set in the build.prop file of your phone/tablet (mine was also set to 32m).

The heapsize determines the total amount of memory a program can use. Memory use by OsmAnd is determined by three factors or a combination of these three factors.

• Information density in a certain region (=information density in that map)

• Complexity of the calculated route

• Length of the calculated route

As said: the heapsize is very often still set to 32m, even for modern 512MB or 1024MB tablets and phones. These phones and tablets perform better when the heapsize is set to a higher value. This heapsize value is a very discriminative factor when calculating routes in OsmAnd.

So I did a few tests with several heap sizes.

I experimented with 24m, 32m (my phone's default as well), 48m, 64m, 80m.

Starting point: My home address, Zwolle, the Netherlands.

Heapsize ->	24m	32m	48m	64m	80m
Siegen, Nordrhein-westfalen, DE (294 km)	OK1	OK	OK	OK	OK
Paris, France (558 km)	X	OK	OK	OK	OK
Praha, Czech republic (809 km)	X	NOK	OK	OK	OK
Saint Malo, Bretagne, France (895 km)	X	NOK	OK	OK	OK
Brest, Bretagne, France (1113 km)	X	X	OK	OK	OK
Bilbao, Spain (1470 km)	X	X	X	OK	OK
Malaga, Andalusia, Spain (2354 km)	X	X	X	X	X2

OK: gets calculated and displayed and OsmAnd continues to function fine.

¹: Keeps functioning but mentions that it doesn't have enough memory to display all screen elements.

NOK (not OK): Sometimes the route gets calculated and displayed but every next action (keypress) will crash OsmAnd.

Most of the time the app crashes during the calculation or at the end of the calculation.

X: OsmAnd crashes during the calculation.

2: Immediately after starting the calculation the error “Empty result”. This is obviously out of limits for OsmAnd and/or android. Most probably due to the 64m memory limit itself unless “android:largeHeap="true"” is used (I'm not a programmer, I can't elaborate on this).

Values of 24m are still the standard on “smaller” and older phones with limited memory.

Note that a value of 48m or 64m for the heapsize is quite acceptable on a 512MB or a 1024MB. A value of 80m or higher should definitely be avoided!

I will write a wiki article about this and I will elaborate a little more on adjusting the “ dalvik.vm.heapsize” in the build.prop in the wiki article. Depending on your phone and used file system (in the phone) this ranges from a piece of cake to almost undoable.

Harry

[Victor Shcherb]

Thanks for investigation, very interesting report indeed! But it is good to mention what is RAM as well, because navigation is done in C++ (native), so it should not be limited by heapsize. Also if you have timings it would be also great!

I will try to memorize this report because it is very valuable. Actually the trips are different by nature,for example there is a very difficult route for calculation Munchen -> Linz (Vienna), even though it is about 200-400 km.

Regards,

Victor

Harry

--
You received this message because you are subscribed to the Google Groups "Osmand" group.
To unsubscribe from this group and stop receiving emails from it, send an email to osmand+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Harry van der Wolf]

Hi,

2013/4/29 Victor Shcherb <victor....@gmail.com>

Thanks for investigation, very interesting report indeed! But it is good to mention what is RAM as well, because navigation is done in C++ (native), so it should not be limited by heapsize. Also if you have timings it would be also great!

You say that the navigation is done in C++. Do you mean the calculation or the navigation, or both?

I did observed that when heap memory is critical (and not just way insufficient), the calculation is running to the end and when it's time to display the application crashes. As if the calculation end correctly correct, but it can't display everything it has calculated.

I do have some numbers but not complete. I now focused on the ability to calculate a route or not.
If I do incorrect assumptions about memory usage, it is because I'm not a programmer. I only reflect on what I observe.

 

I will try to memorize this report because it is very valuable. Actually the trips are different by nature,for example there is a very difficult route for calculation Munchen -> Linz (Vienna), even though it is about 200-400 km.

Yes, I know. That's also why I mentioned the complexity of a route. And also why I used Siegen in Germany. To get to Siegen (from where I live) you have to cross a part of Germany (Ruhrgebiet) that is very densely populated and industrialized, having an extremely dense and complex highway infrastructure and urbanised areas.

Harry

[Victor Shcherb]

I mean only *calculation*. You also could refer how much memory is used in General Settings (enable Development plugin).

If you managed to calculate the complex route, you can switch to aLogcat program and check some interesting info. How much tiles were used (memory), how many times were tiles loaded/unloaded and how many segments were visited (it also gives precise information about timing loading data from file vs calculation time).

I could explain it if it is interesting or unclear.

Victor

--

[Harry van der Wolf]

2013/4/30 Victor Shcherb <victor....@gmail.com>

I mean only *calculation*. You also could refer how much memory is used in General Settings (enable Development plugin).

If you managed to calculate the complex route, you can switch to aLogcat program and check some interesting info. How much tiles were used (memory), how many times were tiles loaded/unloaded and how many segments were visited (it also gives precise information about timing loading data from file vs calculation time).

I could explain it if it is interesting or unclear.

Victor

I understand what you mean and I will have a look at it, but it will take time.

Harry

[Harry van der Wolf]

Hi Victor,

Well, I do have a question after all. How do you want to have the logcat output? Just the complete output?
I did see messages beginning with "I/System.out" and "I/net.osmand". Do you want met to filter on that?

I can/will share timings but I will send the logcat output directly to you as it also contains personal information.

Harry

2013/4/30 Harry van der Wolf <hvd...@gmail.com>

[Harry van der Wolf]

Sorry,

I'm not so structured today. Do you want logs from calculations with enough memory and without enough memory?
I did not check yet, but I assume that switching on the debug plugin will degrade performance and thereby negatively influence timings.

[Victor Shcherb]

Well actually logs will not change results :) They will just clarify some details (make the routes more clear tecnhically)

The most interesting debug info ( for me). 

Apr 30, 2013 10:34:05 AM net.osmand.router.BinaryRoutePlannerOld printInfo

Timings 

Time to calculate : 4.082293, time to load : 0.0, time to load headers : 2.110929

Apr 30, 2013 10:34:05 AM net.osmand.router.BinaryRoutePlannerOld printInfo

How much map tiles do we load

WARNING: Current loaded tiles : 10, maximum loaded tiles 10

Apr 30, 2013 10:34:05 AM net.osmand.router.BinaryRoutePlannerOld printInfo

How much do we load twice/triple

WARNING: Loaded tiles 10 (distinct 10), unloaded tiles 0, loaded more than once same tiles 0

Apr 30, 2013 10:34:05 AM net.osmand.router.BinaryRoutePlannerOld printInfo

How much roads/segments do we visit

WARNING: Visited roads, 8, relaxed roads 0

Apr 30, 2013 10:34:05 AM net.osmand.router.BinaryRoutePlannerOld printInfo

WARNING: Priority queues sizes : 19/21

Apr 30, 2013 10:34:05 AM net.osmand.router.BinaryRoutePlannerOld printInfo

WARNING: Visited segments sizes: 56/39

Routing calculated time distance 55.449043

Victor

[Harry van der Wolf]

Just as an example.
In Osmand I set in the plugins manager both the "OSM editing" and the "OsmAnd development" to On.
In the"Define view -> configure screen" settings I set "osm bugs" to On.

On my laptop I connected my phone via USB
In a sudo bash shell I set: export ANDROID_LOG_TAGS="net.osmand:* *:S"
(to catch all messages from osmand and to suppress all other messages.)
Then I did a "adb logcat -c" to flush my android logs
Then I did a "adb logcat -v time 2>&1 | tee osmand01.log" immediately before starting the application.
I set destination to Bilbao, Spain.

Please find attached a log.
Is this what you are looking for?
Note that I had to outcomment the "ro.kernel.android.checkjni=0" to "#ro.kernel.android.checkjni=0" in my build.prop to get more debug info.
I always have this set to 0 as it improves the speed and performance of your phone/tablet and reduces memory usage.
However: I still don't get the messages you are referring to in your mail.

I do get these overlays displaying what has been done and how much time it costs. Are they saved as well and where?

I can of course log more calculations to the log but I want to whether this is waht you are looking for especially as I can'tfind the specific info you write in your log.

Harry

2013/4/30 Victor Shcherb <victor....@gmail.com>

[Victor Shcherb]

Hm... Not really, the most interesting bit is missing. Here what I found .

04-30 12:52:28.069 W/net.osmand( 2914): RouteProvider Use 41 MB Free 1.8125992 of 21.847626 max 64.0

04-30 12:56:37.199 I/net.osmand( 2914): RouteProvider Finding route contained 12428 points for 249334 ms

And that doesn't contain route debug information, will try to calculation myself to find exact messages/direction in the log.

[Harry van der Wolf]

How do you run your debug sessions?
Which settings in Osmand?
Which settings in Android?
Which settings in ADB?
Note also that I run the OsmAnd+ 1.2.1beta from the PlayStore. Is that one compiled with or without debug info. Did you expect me to use a "latest-build" version? If so: is that one compiled with (more) debug info?

Harry

2013/5/1 Victor Shcherb <victor....@gmail.com>

[Victor Shcherb]

Sorry for delay (lost in emails)

I tested on the latest night build and there is such information there.

Here is what you can see (on log level INFO)

05-18 12:18:48.074: I/net.osmand(25436): BinaryRoutePlannerOld Route segment found 7379428 Piarcoplein

05-18 12:18:48.511: W/net.osmand:native(25436): Start point was found 7366531 [Native]

05-18 12:18:48.535: W/net.osmand:native(25436): End point was found 7366603 [Native]

05-18 12:18:48.543: W/net.osmand:native(25436): [Native] Result visited (visited roads 22, visited segments 39 / 26 , queue sizes 10 / 13 ) 

05-18 12:18:48.543: W/net.osmand:native(25436): [Native] Result timing (time to load 0, time to calc 0, loaded tiles 11) 

05-18 12:18:48.543: W/net.osmand:native(25436): [Native] Memory occupied (Routing context 98 Kb, search 1 Kb)

05-18 12:18:48.543: I/net.osmand:native(25436): Routing calculated time distance 62.161316

05-18 12:18:48.800: W/net.osmand:native(25436): Start point was found 7366603 [Native]

05-18 12:18:48.902: W/net.osmand:native(25436): End point was found 7379428 [Native]

05-18 12:18:51.312: W/net.osmand:native(25436): [Native] Result visited (visited roads 2773, visited segments 5182 / 4121 , queue sizes 306 / 416 ) 

05-18 12:18:51.312: W/net.osmand:native(25436): [Native] Result timing (time to load 3000, time to calc 3000, loaded tiles 344) 

05-18 12:18:51.351: W/net.osmand:native(25436): [Native] Memory occupied (Routing context 3424 Kb, search 151 Kb)

05-18 12:18:51.386: I/net.osmand:native(25436): Routing calculated time distance 949.572876

05-18 12:18:52.129: I/dalvikvm(25436): Jit: resizing JitTable from 8192 to 16384

05-18 12:18:52.359: I/System.out(25436): ROUTE : 

05-18 12:18:52.359: I/System.out(25436): <test regions="" description="" best_percent="" vehicle="car" 

05-18 12:18:52.359: I/System.out(25436):     start_lat="52.2912168" start_lon="4.830735" target_lat="52.38995361328125" target_lon="4.839606285095215" loadedTiles = "406" visitedSegments = "2795" complete_distance = "18197.924" complete_time = "925.22174" routing_time = "1011.7342"  >

05-18 12:18:52.367: I/System.out(25436): <segment id="7366531" start="1" end="2" time = "1.5292176" name = "Sint Urbanusstraat" maxspeed = "29.999998" residential distance = "12.74348" turn = "Go ahead" turn_angle = "0.0" start_bearing = "-65.27885" end_bearing = "-65.27885" description = "Go ahead and go 12.74 meters" />

Victor