SRTM data on Osmand: Vector and hillshade

[yvecai]

On 01/28/12 09:45, Hardy wrote:
> Ok, now since that is sorted out, I would like to summarize my feeling
> that some type of vector mechanism is likely the technically best and
> most future-proof way to go here.
>
> A huge number of people go for tiles because they want countour or
> hillshade information. So this is what we should continue working on:
> - Improve and advertise the contour vector add-ons (SRTM files: They
> are available on Yves' web site and supported in my default
> renderer!). But availability in Osmands Download list is still
> marginal, and there are a few implementation bugs or improvements
> discussed elsewhere (occasionally out of memory, "vortex issue",
> occasional gaps)
> - Maybe think abou a way to have a similar way of putting some
> hillshade-info as a vector add-on?
>
> Best regards,
> Hardy
Let's summarize:
Vector contour line (or isohypse) files by country are here:
http://www.pistes-nordiques.org/downloads
Or in this directory:
http://www.pistes-nordiques.org/download/Osmand-contours/

Does somebody can allow me to simply scp these files to Osmand's
downloads? Or does somebody can wget them from Osmand download ? This is
22GB of files, America (north and south), Africa and Europe are complete.
Same for world-ski.obf here:
http://www.pistes-nordiques.org/download/world-ski.obf.zip

Now the process is stuck for two reasons: First I have to understand why
my process is not working for Yemen, secondly my server is busy in
winter for updating nordic ski pistes daily (and all pistes for Osmand
by the way). Thus I will work again this spring to complete the file set
when the snow have melt and mappers less active in the field.

Improvement: Improving voids in SRTM data is not really possible, we
have to wait for another set of data available with a suitable license.
Interpolating is not an option here, we can't invent the relief just to
look nice.
Borders artifacts: contour lines are cropped by osmfilter, this tools
only remove the nodes outside of the cropping polygon. The good way to
go would be to split the contour into several ways when it cross the
border before. The tool is in C, Markus Weber would certainly welcome a
patch. Alternatively, this could be handled by a plugin for Osmosis (in
Java) which is also able to cut dataset along polygons.

Hillshading
I shared to some people here a complete set of SRTM hillshading tiles
from z0 to z11. I'm currently checking how this set can be resampled for
higher zoom directly by Osmand.
We can use these files as a sqlitedb, or even rendering hillshade form
raw SRTM but the difficulty is to distribute them. An extract per
country available via Osmand download seems the easiest way to go.
Polygon files are available from cloudmade, but we need a tool to cut
the big sqlite with them. Any idea?

Yves

[Shorty66]

id love directly rendered hillshades from raw srtm data in osmand.
Perhaps it would be even possible to render contourlines from raw srtm
data?

There would be many cool things to do with raw srtm data like a new
routing function for the least strenous way (low inclination)...

[yvecai]

Not from raw SRTM yet, but some code to test and questions here:
http://code.google.com/p/osmand/issues/detail?id=858&thanks=858&ts=1327958570

Yves

[andre van atten]

Hi,

This is good progress! If resampling of hillshade tiles is possible, it can be used at higher zoomlevels, and can be combined with vector contourlines in some overlapping levels. Hope you can overcome the memory issues and the edge effect. After that we should look into creating hillshade tile packages. I created some samples already on the osmati server.

Andre

Op 30 jan. 2012 22:24 schreef "yvecai" <yve...@gmail.com> het volgende:

[yvecai]

The resampling of sqlite tiles is working:

https://github.com/yvecai/Osmand/tree/hillshade-resample

Please test it, if you are pleased with it, I'll try to add it as an option selectable via the setting menu, or maybe by default?

Yves

[Victor Shcherb]

Now that fork is aware of jenkins.osmand.net system!

http://download.osmand.net/night-builds/?C=M;O=D 

http://download.osmand.net/night-builds/OsmAndMapCreator-hillshade-resample-nb-2012-02-07.zip

https://jenkins.osmand.net/job/OsmAnd/712/console 

Victor

2012/2/7 yvecai <yve...@gmail.com>

[yvecai]

Cool! That's easy for testers.
But ... the only modification I've made are on Osmand, not OsmandMapCreator :))
Yves

[andre van atten]

Hi Yves,

I tested with a srtm take package containing levels 4 - 11. Levels 12 - 16 look perfect, but CPU time takes longer on every zoom in.. Level 17 is too much: it destroys the canvas, and osmand goes back to the search form I used before, thereby releasing about 25 a 30 mb of RAM. Strange, osmand did not crash, but loosed canvas. So maybe some kind of overzoom protection could be added, simply switch off the overlay. However, main point is: it looks very good!
By the way, could this mechanism be used when zooming in one- color tiles like water or grassland, so they don't have to be loaded from tile server?

Andre

Op 7 feb. 2012 22:11 schreef "yvecai" <yve...@gmail.com> het volgende:

[Jindřich Makovička]

From what I am seeing, the code for getMetaTile always retrieves and
processes 9 database tiles per 1 "virtual" tile, which is certainly a
waste of resources. At the very least, the code should distinguish
between virtual tiles inside of the real tile, and tiles on the edge,
and retrieve 1 db tile for the inner tile, and all neighbors for the
virtual tiles at the edge of real ones. Ideally, only 4 real tiles at
maximum per one virtual tile should be needed - 4 for corners, 2 for
edges, 1 for the rest:

eg. number of needed db tiles for for 4x scaling

4 2 2 4
2 1 1 2
2 1 1 2
4 2 2 4

For higher zoom levels, this should bring a huge speedup, as most of
the virtual tiles are inside.

--
Jindřich Makovička

[Victor Shcherb]

Sounds very strange. I don't agree that it should be combined in kind of virtual tile. Virtual tile should be rectangular form and if the map is rotated it will load more tiles than needed to display and will cause 

Also you didn't take into account that screen can be different and 4 tiles will not be enough, if you are going to download many virtual tiles what is the difference between current situation. 

That solution only make code more complicated and doesn't produce big performance win.

Note : allocating big chunk of memory once per time introduce different problems. I need to review original solution how it solves out of memory exception and how it updates number of loaded tiles in ResourceManager.

Thanks,

Victor

среда, 8 февраля 2012 г. пользователь Jindřich Makovička писал:

>>>> > > > Ok, now since that is sorted out, I would--
Jindřich Makovička

[Yves]

Jindrich, you're right I thought about it.

"Jindřich Makovička" <mako...@gmail.com> a écrit :

[Yves]

Viktor, the metatile is there behind the scene to avoid resampling artifacts. ResourceManager asks for one tile and receive one tile.
Yves

Victor Shcherb <victor....@gmail.com> a écrit :

[Victor Shcherb]

Sorry I think I misunderstand the topic. I was just saying to keep 256x256 tiles on screen not to produce 1024x1024 (that is very bad for rotating map).

Sorry for confusion.

среда, 8 февраля 2012 г. пользователь Yves писал:

[Jindřich Makovička]

Hi,

I pushed a rewritten and simplified (but still not optimized) version
to my repo. It's equivalent in function, but much shorter.

https://github.com/jmakovicka/Osmand/tree/hillshade-resample

--
Jindřich Makovička

[Jindřich Makovička]

Just added the optimization I mentioned in my previous post, in the same branch.

2012/2/8 Jindřich Makovička <mako...@gmail.com>:

--
Jindřich Makovička

[yvecai]

Very nice job! I wish I understand bitwise operators ;-)
I commited your change on the freshly created branch:
https://github.com/osmandapp/Osmand/tree/hillshade-resample

Sqlite resampling is available here
http://download.osmand.net/night-builds/?C=M;O=D for testing.

Yves

Le 08/02/2012 22:27, Jind�ich Makovi�ka a �crit :

> Just added the optimization I mentioned in my previous post, in the same branch.
>

> 2012/2/8 Jind�ich Makovi�ka<mako...@gmail.com>:

>> Hi,
>>
>> I pushed a rewritten and simplified (but still not optimized) version
>> to my repo. It's equivalent in function, but much shorter.
>>
>> https://github.com/jmakovicka/Osmand/tree/hillshade-resample
>>
>> On Wed, Feb 8, 2012 at 10:06, Yves<yve...@gmail.com> wrote:
>>> Jindrich, you're right I thought about it.
>>>
>>>
>>>

>>> "Jind�ich Makovi�ka"<mako...@gmail.com> a �crit :

>>>>>> Le 07/02/2012 21:49, Victor Shcherb a �crit :

>>>>>>
>>>>>> Now that fork is aware of jenkins.osmand.net system!
>>>>>> http://download.osmand.net/night-builds/?C=M;O=D
>>>>>>
>>>>>>
>>>>>> http://download.osmand.net/night-builds/OsmAndMapCreator-hillshade-resample-nb-2012-02-07.zip
>>>>>> https://jenkins.osmand.net/job/OsmAnd/712/console
>>>>>>
>>>>>> Victor
>>>>>>
>>>>>> 2012/2/7 yvecai<yve...@gmail.com>
>>>>>>> The resampling of sqlite tiles is working:
>>>>>>>
>>>>>>> https://github.com/yvecai/Osmand/tree/hillshade-resample
>>>>>>>
>>>>>>> Please test it, if you are pleased with it, I'll try to add it as an
>>>>>>> option selectable via the setting menu, or maybe by default?
>>>>>>>
>>>>>>> Yves
>>>>>>>

>>>>>>> Le 01/02/2012 23:18, andre van atten a �crit :

>>>> Jind�ich Makovi�ka
>>
>>
>> --
>> Jind�ich Makovi�ka
>
>

[Jindřich Makovička]

I hope it's not too complicated... Most of bit shifts are just
replacements of Math.pow(2,n), and the flags bitfield contains 1 an
used and 0 for an unused DB tile. in the row/column order.

I have currently installed the last version from my branch, with
following small (but mportant) changes still uncommitted:

1) margin set to 1 px to further lower memory footprint
2) all calls to System.gc() removed - adding even more of them was a
bad idea, hammering GC just slows things down and eventually made my
Evo 3D reboot.

Also, the last commit limits the maximum zoom to 15, because when
interpolating 4x4 tile to 256x256, it just results in some random
blobs with no particular connection to reality, from what I have seen.

Still need to check if the version without System.gc() works on the
emulator with lower heap size, but I am happy with it on my mobile -
it is pretty fast and stable so far.

2012/2/9 yvecai <yve...@gmail.com>:

> Very nice job! I wish I understand bitwise operators ;-)
> I commited your change on the freshly created branch:
> https://github.com/osmandapp/Osmand/tree/hillshade-resample
>
> Sqlite resampling is available here
> http://download.osmand.net/night-builds/?C=M;O=D for testing.
>
> Yves
>

> Le 08/02/2012 22:27, Jindřich Makovička a écrit :
>
>> Just added the optimization I mentioned in my previous post, in the same
>> branch.
>>

>> 2012/2/8 Jindřich Makovička<mako...@gmail.com>:

>>>
>>> Hi,
>>>
>>> I pushed a rewritten and simplified (but still not optimized) version
>>> to my repo. It's equivalent in function, but much shorter.
>>>
>>> https://github.com/jmakovicka/Osmand/tree/hillshade-resample
>>>
>>> On Wed, Feb 8, 2012 at 10:06, Yves<yve...@gmail.com>  wrote:
>>>>
>>>> Jindrich, you're right I thought about it.
>>>>
>>>>
>>>>

>>>> "Jindřich Makovička"<mako...@gmail.com>  a écrit :
>>>>>

[andre van atten]

Latest version zooms in on srtm package more easy from levels 11-16, but when zooming to level 17 on a place where 4 tiles do touch eachother, I had a crash of Osmand. It seems more stable though then the previous version
I use high resolution and native rendering

Andre

[yvecai]

Le 09/02/2012 10:31, Jind�ich Makovi�ka a �crit :

> I hope it's not too complicated... Most of bit shifts are just
> replacements of Math.pow(2,n), and the flags bitfield contains 1 an
> used and 0 for an unused DB tile. in the row/column order.
>
> I have currently installed the last version from my branch, with
> following small (but mportant) changes still uncommitted:
>
> 1) margin set to 1 px to further lower memory footprint
> 2) all calls to System.gc() removed - adding even more of them was a
> bad idea, hammering GC just slows things down and eventually made my
> Evo 3D reboot.

I think it's worth to get rid of tile used in getmetatile(), a
bitmap.recycle() is sufficient, no?

> Also, the last commit limits the maximum zoom to 15, because when
> interpolating 4x4 tile to 256x256, it just results in some random
> blobs with no particular connection to reality, from what I have seen.

Depends where, SRTM dataset are not 'looking' alike everywhere. Here I
find it usefull up to zoom 20. Ok maybe not that usefull, but it's not
wrong :)

> Still need to check if the version without System.gc() works on the
> emulator with lower heap size, but I am happy with it on my mobile -
> it is pretty fast and stable so far.
> 2012/2/9 yvecai<yve...@gmail.com>:
>> Very nice job! I wish I understand bitwise operators ;-)
>> I commited your change on the freshly created branch:
>> https://github.com/osmandapp/Osmand/tree/hillshade-resample
>>
>> Sqlite resampling is available here
>> http://download.osmand.net/night-builds/?C=M;O=D for testing.
>>
>> Yves
>>

>> Le 08/02/2012 22:27, Jind�ich Makovi�ka a �crit :
>>

>>> Just added the optimization I mentioned in my previous post, in the same
>>> branch.
>>>

>>> 2012/2/8 Jind�ich Makovi�ka<mako...@gmail.com>:

>>>> Hi,
>>>>
>>>> I pushed a rewritten and simplified (but still not optimized) version
>>>> to my repo. It's equivalent in function, but much shorter.
>>>>
>>>> https://github.com/jmakovicka/Osmand/tree/hillshade-resample
>>>>
>>>> On Wed, Feb 8, 2012 at 10:06, Yves<yve...@gmail.com> wrote:
>>>>> Jindrich, you're right I thought about it.
>>>>>
>>>>>
>>>>>

>>>>> "Jind�ich Makovi�ka"<mako...@gmail.com> a �crit :

>>>>>>>> Le 07/02/2012 21:49, Victor Shcherb a �crit :

>>>>>>>>
>>>>>>>> Now that fork is aware of jenkins.osmand.net system!
>>>>>>>> http://download.osmand.net/night-builds/?C=M;O=D
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> http://download.osmand.net/night-builds/OsmAndMapCreator-hillshade-resample-nb-2012-02-07.zip
>>>>>>>> https://jenkins.osmand.net/job/OsmAnd/712/console
>>>>>>>>
>>>>>>>> Victor
>>>>>>>>
>>>>>>>> 2012/2/7 yvecai<yve...@gmail.com>
>>>>>>>>> The resampling of sqlite tiles is working:
>>>>>>>>>
>>>>>>>>> https://github.com/yvecai/Osmand/tree/hillshade-resample
>>>>>>>>>
>>>>>>>>> Please test it, if you are pleased with it, I'll try to add it as an
>>>>>>>>> option selectable via the setting menu, or maybe by default?
>>>>>>>>>
>>>>>>>>> Yves
>>>>>>>>>

>>>>>>>>> Le 01/02/2012 23:18, andre van atten a �crit :

[Jindřich Makovička]

I am not sure how zoom 20 could work for you at all. For the sqlite
file (http://www.pistes-nordiques.org/download/switzerland.sqlitedb)
you posted in the google code issue, last zoom level that has
theoretically any sense is 18, which has 1 original _pixel_ per tile.
With original margin of 6 pixels, it would then happily expand to
3328x3328 bitmap :)

Disregard the above if you used some other sqlite db with a higher zoom level.

I also updated my branch with the mentioned changes.

2012/2/9 yvecai <yve...@gmail.com>:

>>> Le 08/02/2012 22:27, Jindřich Makovička a écrit :
>>>
>>>> Just added the optimization I mentioned in my previous post, in the same
>>>> branch.
>>>>

>>>> 2012/2/8 Jindřich Makovička<mako...@gmail.com>:

>>>>>
>>>>> Hi,
>>>>>
>>>>> I pushed a rewritten and simplified (but still not optimized) version
>>>>> to my repo. It's equivalent in function, but much shorter.
>>>>>
>>>>> https://github.com/jmakovicka/Osmand/tree/hillshade-resample
>>>>>
>>>>> On Wed, Feb 8, 2012 at 10:06, Yves<yve...@gmail.com>    wrote:
>>>>>>
>>>>>> Jindrich, you're right I thought about it.
>>>>>>
>>>>>>
>>>>>>

>>>>>> "Jindřich Makovička"<mako...@gmail.com>    a écrit :
>>>>>>>

[yvecai]

I tried with an AVD with 16M heap size: resampling is completely
worthless on a G1 !
I added exception handling, at least it should not crash anymore, just
it won't display tiles if it can't resample them. 24M heap size strongly
advised !
The build is updated.
Yves

Le 09/02/2012 19:49, Jind�ich Makovi�ka a �crit :

> I am not sure how zoom 20 could work for you at all. For the sqlite
> file (http://www.pistes-nordiques.org/download/switzerland.sqlitedb)
> you posted in the google code issue, last zoom level that has
> theoretically any sense is 18, which has 1 original _pixel_ per tile.
> With original margin of 6 pixels, it would then happily expand to
> 3328x3328 bitmap :)
>
> Disregard the above if you used some other sqlite db with a higher zoom level.
>
> I also updated my branch with the mentioned changes.
>
> 2012/2/9 yvecai<yve...@gmail.com>:

>> Le 09/02/2012 10:31, Jind�ich Makovi�ka a �crit :
>>

>>>> Le 08/02/2012 22:27, Jind�ich Makovi�ka a �crit :

>>>>
>>>>> Just added the optimization I mentioned in my previous post, in the same
>>>>> branch.
>>>>>

>>>>> 2012/2/8 Jind�ich Makovi�ka<mako...@gmail.com>:

>>>>>> Hi,
>>>>>>
>>>>>> I pushed a rewritten and simplified (but still not optimized) version
>>>>>> to my repo. It's equivalent in function, but much shorter.
>>>>>>
>>>>>> https://github.com/jmakovicka/Osmand/tree/hillshade-resample
>>>>>>
>>>>>> On Wed, Feb 8, 2012 at 10:06, Yves<yve...@gmail.com> wrote:
>>>>>>> Jindrich, you're right I thought about it.
>>>>>>>
>>>>>>>
>>>>>>>

>>>>>>> "Jind�ich Makovi�ka"<mako...@gmail.com> a �crit :

>>>>>>>>>> Le 07/02/2012 21:49, Victor Shcherb a �crit :

>>>>>>>>>>
>>>>>>>>>> Now that fork is aware of jenkins.osmand.net system!
>>>>>>>>>> http://download.osmand.net/night-builds/?C=M;O=D
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> http://download.osmand.net/night-builds/OsmAndMapCreator-hillshade-resample-nb-2012-02-07.zip
>>>>>>>>>> https://jenkins.osmand.net/job/OsmAnd/712/console
>>>>>>>>>>
>>>>>>>>>> Victor
>>>>>>>>>>
>>>>>>>>>> 2012/2/7 yvecai<yve...@gmail.com>
>>>>>>>>>>> The resampling of sqlite tiles is working:
>>>>>>>>>>>
>>>>>>>>>>> https://github.com/yvecai/Osmand/tree/hillshade-resample
>>>>>>>>>>>
>>>>>>>>>>> Please test it, if you are pleased with it, I'll try to add it as
>>>>>>>>>>> an
>>>>>>>>>>> option selectable via the setting menu, or maybe by default?
>>>>>>>>>>>
>>>>>>>>>>> Yves
>>>>>>>>>>>

>>>>>>>>>>> Le 01/02/2012 23:18, andre van atten a �crit :

[Victor Shcherb]

One moment guys!

24M is only for good devices, for my one (HTC Hero) still 16M. And for budget as well.

Don't hurry up :)

2012/2/9 yvecai <yve...@gmail.com>

I tried with an AVD with 16M heap size: resampling is completely worthless on a G1 !
I added exception handling, at least it should not crash anymore, just it won't display tiles if it can't resample them. 24M heap size strongly advised !
The build is updated.
Yves

Le 09/02/2012 19:49, Jindřich Makovička a écrit :

I am not sure how zoom 20 could work for you at all. For the sqlite
file (http://www.pistes-nordiques.org/download/switzerland.sqlitedb)
you posted in the google code issue, last zoom level that has
theoretically any sense is 18, which has 1 original _pixel_ per tile.
With original margin of 6 pixels, it would then happily expand to
3328x3328 bitmap :)

Disregard the above if you used some other sqlite db with a higher zoom level.

I also updated my branch with the mentioned changes.

2012/2/9 yvecai<yve...@gmail.com>:

2012/2/8 Jindřich Makovička<mako...@gmail.com>:

Hi,

I pushed a rewritten and simplified (but still not optimized) version
to my repo. It's equivalent in function, but much shorter.

https://github.com/jmakovicka/Osmand/tree/hillshade-resample

On Wed, Feb 8, 2012 at 10:06, Yves<yve...@gmail.com>      wrote:

Jindrich, you're right I thought about it.

"Jindřich Makovička"<mako...@gmail.com>      a écrit :

 From what I am seeing, the code for getMetaTile always retrieves and

Jindřich Makovička

--
Jindřich Makovička

[yvecai]

I think we must miss something about bitmaps handling, any idea on the direction to take Viktor?
Yves

Le 09/02/2012 23:36, Victor Shcherb a �crit�:

One moment guys!

24M is only for good devices, for my one (HTC Hero) still 16M. And for budget as well.

Don't hurry up :)

2012/2/9 yvecai <yve...@gmail.com>

I tried with an AVD with 16M heap size: resampling is completely worthless on a G1 !
I added exception handling, at least it should not crash anymore, just it won't display tiles if it can't resample them. 24M heap size strongly advised !
The build is updated.
Yves

Le 09/02/2012 19:49, Jind�ich Makovi�ka a �crit :

I am not sure how zoom 20 could work for you at all. For the sqlite
file (http://www.pistes-nordiques.org/download/switzerland.sqlitedb)
you posted in the google code issue, last zoom level that has
theoretically any sense is 18, which has 1 original _pixel_ per tile.
With original margin of 6 pixels, it would then happily expand to
3328x3328 bitmap :)

Disregard the above if you used some other sqlite db with a higher zoom level.

I also updated my branch with the mentioned changes.

2012/2/9 yvecai<yve...@gmail.com>:

Le 09/02/2012 10:31, Jind�ich Makovi�ka a �crit :

Le 08/02/2012 22:27, Jind�ich Makovi�ka a �crit :

Just added the optimization I mentioned in my previous post, in the same
branch.

2012/2/8 Jind�ich Makovi�ka<mako...@gmail.com>:

Hi,

I pushed a rewritten and simplified (but still not optimized) version
to my repo. It's equivalent in function, but much shorter.

https://github.com/jmakovicka/Osmand/tree/hillshade-resample

On Wed, Feb 8, 2012 at 10:06, Yves<yve...@gmail.com> � � �wrote:

Jindrich, you're right I thought about it.

"Jind�ich Makovi�ka"<mako...@gmail.com> � � �a �crit :

�From what I am seeing, the code for getMetaTile always retrieves and

Op 7 feb. 2012 22:11 schreef "yvecai"<yve...@gmail.com> � � �het
volgende:

Cool! That's easy for testers.
But ... the only modification I've made are on Osmand, not
OsmandMapCreator :))
Yves

Le 07/02/2012 21:49, Victor Shcherb a �crit :

Now that fork is aware of jenkins.osmand.net system!
http://download.osmand.net/night-builds/?C=M;O=D




http://download.osmand.net/night-builds/OsmAndMapCreator-hillshade-resample-nb-2012-02-07.zip
https://jenkins.osmand.net/job/OsmAnd/712/console

Victor

2012/2/7 yvecai<yve...@gmail.com>

The resampling of sqlite tiles is working:

https://github.com/yvecai/Osmand/tree/hillshade-resample

Please test it, if you are pleased with it, I'll try to add it as
an
option selectable via the setting menu, or maybe by default?

Yves

Le 01/02/2012 23:18, andre van atten a �crit :

Hi,

This is good progress! If resampling of hillshade tiles is
possible,
it
can be used at higher zoomlevels, and can be combined with vector
contourlines in some overlapping levels. Hope you can overcome the
memory
issues and the edge effect. After that we should look into
creating
hillshade tile packages. I created some samples already on the
osmati
server.

Andre

Op 30 jan. 2012 22:24 schreef "yvecai"<yve...@gmail.com> � � �het
volgende:

Not from raw SRTM yet, but some code to test and questions here:




http://code.google.com/p/osmand/issues/detail?id=858&thanks=858&ts=1327958570

Yves

On Jan 29, 9:31 pm, Shorty66<chrissik...@googlemail.com>

�wrote:

id love directly rendered hillshades from raw srtm data in
osmand.
Perhaps it would be even possible to render contourlines from
raw
srtm
data?

There would be many cool things to do with raw srtm data like a
new

routing function for the least strenous way (low inclination)...

On 28 Jan., 17:19, yvecai<yve...@gmail.com> � � �wrote:

On 01/28/12 09:45, Hardy wrote:

Ok, now since that is sorted out, I would like to summarize my
feeling
that some type of vector mechanism is likely the technically
best
and
most future-proof way to go here.
A huge number of people go for tiles because they want
countour
or
hillshade information. So this is what we should continue
working
on:
- Improve and advertise the

contour vector add-ons (SRTM files:

They
are available on Yves' web site and supported in my default
renderer!). But availability in Osmands Download list is still

marginal, and there are �a few implementation bugs or

--
Jind�ich Makovi�ka

[Jindřich Makovička]

I am afraid there is no silver bullet. If you change the code to
return the original tile the same way as in the unscaled case, it
still crashes with 16 or 24MB heap.

I played a little bit with bitmap scaling using Matrix, which further
simplifies the code and maybe lowers memory consumption a little bit,
but no change in stability.

2012/2/9 yvecai <yve...@gmail.com>:

> I think we must miss something about bitmaps handling, any idea on the
> direction to take Viktor?
> Yves
>

> Le 09/02/2012 23:36, Victor Shcherb a écrit :
>
> One moment guys!
> 24M is only for good devices, for my one (HTC Hero) still 16M. And for
> budget as well.
>
> Don't hurry up :)
>
> 2012/2/9 yvecai <yve...@gmail.com>
>>
>> I tried with an AVD with 16M heap size: resampling is completely worthless
>> on a G1 !
>> I added exception handling, at least it should not crash anymore, just it
>> won't display tiles if it can't resample them. 24M heap size strongly
>> advised !
>> The build is updated.
>> Yves
>>

>>>>>>> 2012/2/8 Jindřich Makovička<mako...@gmail.com>:

>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I pushed a rewritten and simplified (but still not optimized)
>>>>>>>> version
>>>>>>>> to my repo. It's equivalent in function, but much shorter.
>>>>>>>>
>>>>>>>> https://github.com/jmakovicka/Osmand/tree/hillshade-resample
>>>>>>>>
>>>>>>>> On Wed, Feb 8, 2012 at 10:06, Yves<yve...@gmail.com>      wrote:
>>>>>>>>>
>>>>>>>>> Jindrich, you're right I thought about it.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>

>>>>>>>>> "Jindřich Makovička"<mako...@gmail.com>      a écrit :
>>>>>>>>>>

>>>>>>>>>>> Op 7 feb. 2012 22:11 schreef "yvecai"<yve...@gmail.com>      het

>>>>>>>>>>>>>  het
>>>>>>>>>>>>> volgende:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Not from raw SRTM yet, but some code to test and questions
>>>>>>>>>>>>>> here:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> http://code.google.com/p/osmand/issues/detail?id=858&thanks=858&ts=1327958570
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yves
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jan 29, 9:31 pm, Shorty66<chrissik...@googlemail.com>

>>>>>>>>>>>>>>  wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> id love directly rendered hillshades from raw srtm data in
>>>>>>>>>>>>>>> osmand.
>>>>>>>>>>>>>>> Perhaps it would be even possible to render contourlines from
>>>>>>>>>>>>>>> raw
>>>>>>>>>>>>>>> srtm
>>>>>>>>>>>>>>> data?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> There would be many cool things to do with raw srtm data like
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>
>>>>>>>>>> routing function for the least strenous way (low inclination)...
>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>> On 28 Jan., 17:19, yvecai<yve...@gmail.com>      wrote:

>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 01/28/12 09:45, Hardy wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ok, now since that is sorted out, I would like to summarize
>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>> feeling
>>>>>>>>>>>>>>>>> that some type of vector mechanism is likely the
>>>>>>>>>>>>>>>>> technically
>>>>>>>>>>>>>>>>> best
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> most future-proof way to go here.
>>>>>>>>>>>>>>>>> A huge number of people go for tiles because they want
>>>>>>>>>>>>>>>>> countour
>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>> hillshade information. So this is what we should continue
>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>> on:
>>>>>>>>>>>>>>>>> - Improve and advertise the
>>>>>>>>>>
>>>>>>>>>> contour vector add-ons (SRTM files:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> They
>>>>>>>>>>>>>>>>> are available on Yves' web site and supported in my default
>>>>>>>>>>>>>>>>> renderer!). But availability in Osmands Download list is
>>>>>>>>>>>>>>>>> still

>>>>>>>>>>>>>>>>> marginal, and there are  a few implementation bugs or

>>>>>>>>>> Jindřich Makovička
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Jindřich Makovička
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>
>>>
>>
>
>

--
Jindřich Makovička

[Jindřich Makovička]

Update: I turned off hires rendering, and changed the emulator display
size to something adequate (QVGA), and 16MB suddenly works well with
my last version.

I still think the OOM exception handling is a good idea though.

2012/2/10 Jindřich Makovička <mako...@gmail.com>:

--
Jindřich Makovička

[yvecai]

This build should work fine on any devices:
http://download.osmand.net/night-builds/OsmAnd-hillshade-resample-nb-2012-02-10.apk
Yves

On Feb 10, 8:58 am, Jindřich Makovička <makov...@gmail.com> wrote:
> Update: I turned off hires rendering, and changed the emulator display
> size to something adequate (QVGA), and 16MB suddenly works well with
> my last version.
>
> I still think the OOM exception handling is a good idea though.
>

> 2012/2/10 Jindøich Makovièka <makov...@gmail.com>:

>
>
>
>
>
>
>
> > I am afraid there is no silver bullet. If you change the code to
> > return the original tile the same way as in the unscaled case, it
> > still crashes with 16 or 24MB heap.
>
> > I played a little bit with bitmap scaling using Matrix, which further
> > simplifies the code and maybe lowers memory consumption a little bit,
> > but no change in stability.
>
> > 2012/2/9 yvecai <yve...@gmail.com>:
> >> I think we must miss something about bitmaps handling, any idea on the
> >> direction to take Viktor?
> >> Yves
>
> >> Le 09/02/2012 23:36, Victor Shcherb a écrit :
>
> >> One moment guys!
> >> 24M is only for good devices, for my one (HTC Hero) still 16M. And for
> >> budget as well.
>
> >> Don't hurry up :)
>
> >> 2012/2/9 yvecai <yve...@gmail.com>
>
> >>> I tried with an AVD with 16M heap size: resampling is completely worthless
> >>> on a G1 !
> >>> I added exception handling, at least it should not crash anymore, just it
> >>> won't display tiles if it can't resample them. 24M heap size strongly
> >>> advised !
> >>> The build is updated.
> >>> Yves
>

> >>> Le 09/02/2012 19:49, Jindøich Makovièka a écrit :
>
> >>>> I am not sure how zoom 20 could work for you at all. For the sqlite
> >>>> file (http://www.pistes-nordiques.org/download/switzerland.sqlitedb)
> >>>> you posted in the google code issue, last zoom level that has
> >>>> theoretically any sense is 18, which has 1 original _pixel_ per tile.
> >>>> With original margin of 6 pixels, it would then happily expand to
> >>>> 3328x3328 bitmap :)
>
> >>>> Disregard the above if you used some other sqlite db with a higher zoom
> >>>> level.
>
> >>>> I also updated my branch with the mentioned changes.
>
> >>>> 2012/2/9 yvecai<yve...@gmail.com>:
>

> >>>>>>>http://download.osmand.net/night-builds/?C=M;O=Dfor testing.
>
> >>>>>>> Yves

>
> >>>>>>> Le 08/02/2012 22:27, Jindøich Makovièka a écrit :
>
> >>>>>>>> Just added the optimization I mentioned in my previous post, in the
> >>>>>>>> same
> >>>>>>>> branch.
>

> >>>>>>>> 2012/2/8 Jindøich Makovièka<makov...@gmail.com>:

>
> >>>>>>>>> Hi,
>
> >>>>>>>>> I pushed a rewritten and simplified (but still not optimized)
> >>>>>>>>> version
> >>>>>>>>> to my repo. It's equivalent in function, but much shorter.
>
> >>>>>>>>>https://github.com/jmakovicka/Osmand/tree/hillshade-resample
>
> >>>>>>>>> On Wed, Feb 8, 2012 at 10:06, Yves<yve...@gmail.com>      wrote:
>
> >>>>>>>>>> Jindrich, you're right I thought about it.
>

> >>>>>>>>>> "Jindøich Makovièka"<makov...@gmail.com>      a écrit :

>
> >>>>>>>>>>>  From what I am seeing, the code for getMetaTile always retrieves
> >>>>>>>>>>> and
> >>>>>>>>>>> processes 9 database tiles per 1 "virtual" tile, which is
> >>>>>>>>>>> certainly a
> >>>>>>>>>>> waste of resources. At the very least, the code should distinguish
> >>>>>>>>>>> between virtual tiles inside of the real tile, and tiles on the
> >>>>>>>>>>> edge,
> >>>>>>>>>>> and retrieve 1 db tile for the inner tile, and all neighbors for
> >>>>>>>>>>> the
> >>>>>>>>>>> virtual tiles at the edge of real ones. Ideally, only 4 real tiles
> >>>>>>>>>>> at
> >>>>>>>>>>> maximum per one virtual tile should be needed - 4 for corners, 2
> >>>>>>>>>>> for
> >>>>>>>>>>> edges, 1 for the rest:
>
> >>>>>>>>>>> eg. number of needed db tiles for for 4x scaling
>
> >>>>>>>>>>> 4 2 2 4
> >>>>>>>>>>> 2 1 1 2
> >>>>>>>>>>> 2 1 1 2
> >>>>>>>>>>> 4 2 2 4
>
> >>>>>>>>>>> For higher zoom levels, this should bring a huge speedup, as most
> >>>>>>>>>>> of
> >>>>>>>>>>> the virtual tiles are inside.
>
> >>>>>>>>>>> On Wed, Feb 8, 2012 at 00:24, andre van

> >>>>>>>>>>> atten<andrevanat...@gmail.com>

> >>>>>>>>>>>>>http://download.osmand.net/night-builds/OsmAndMapCreator-hillshade-re...

> >>>>>>>>>>>>>>>http://code.google.com/p/osmand/issues/detail?id=858&thanks=858&ts=13...

>
> >>>>>>>>>>>>>>> Yves
>
> >>>>>>>>>>>>>>> On Jan 29, 9:31 pm, Shorty66<chrissik...@googlemail.com>
> >>>>>>>>>>>>>>>  wrote:
>
> >>>>>>>>>>>>>>>> id love directly rendered hillshades from raw srtm data in
> >>>>>>>>>>>>>>>> osmand.
> >>>>>>>>>>>>>>>> Perhaps it would be even possible to render contourlines from
> >>>>>>>>>>>>>>>> raw
> >>>>>>>>>>>>>>>> srtm
> >>>>>>>>>>>>>>>> data?
>
> >>>>>>>>>>>>>>>> There would be many cool things to do with raw srtm data like
> >>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>> new
>
> >>>>>>>>>>> routing function for the least strenous way (low inclination)...
>
> >>>>>>>>>>>>>>>> On 28 Jan., 17:19, yvecai<yve...@gmail.com>      wrote:
>
> >>>>>>>>>>>>>>>>> On 01/28/12 09:45, Hardy wrote:
>

> ...
>
> read more »

[andre van atten]

Hi Yves and Jindřich,

I can confirm that build 753, 10 Feb, works fine now, even hires. You guys are fantastic, this is a good showcase again that opensource stimulates working together and sharpening each other.
We should distribute srtm regions now , so they become visible in download manager. We should not use srtm aa name, but hillshade or height maps
Western europe is about 180 mb, levels 4 - 11. Makes many people happy to be able to use it as overlay

Andre

[yvecai]

The best thing to do would be to make extracts by country, so that these
extract are easy to find from the offline download menu.
That's a job that could be done manually (ie. crowdsourced) in Mobac,
it's quite fast to draw a coarse polygon around a country and extract an
osmand sqlite from there.
But we need a solution for crowdsourcing it. Eventually, a few person
could do the job, any volonteer to take one continent? Could you make
big sqlites by continents available somewhere, Andre? Then you can give
the download link to somebody if it process all the countries in the
continent and upload back.

Yves

[yvecai]

Update on contours:
I'm able to extract contour lines in shapefile format without the
artifacts at countries' borders, I just query the postgis DB that store
the contour lines on my website.
There is still some work to do for these big files to be handled to
OsmandMapCreator though,but it's on it's way.

I take the opportunity to ask if anybody have a couple of minute to
spend to manually download polygon files from cloudmade. They are
available, but unfortunately they don't want to send me a package. I
will need all asia and ocenia.
This is the .poly files here http://downloads.cloudmade.com/
For big countries, prefer sub-areas (like Russia, China and Australia)
otherwise it won't pass.

Yves

[andre van atten]

Hi,

Time is not the thing I have plenty of ;-) and processing and uploading 30 gb of Srtm will take lots of time I am afraid. There is already a srtm of western Europe 200 mb and Europe 1.8 gb on osmati downloads, I will see if I can process some more continents up to 2 gb per package next weeks. Please be patient, I have to make free space first.

Andre

Op 12 feb. 2012 17:18 schreef "yvecai" <yve...@gmail.com> het volgende:

[yvecai]

What did you used to make the Europe sqlite from the tile I provided, MOBAC?
I didn't success to do it.
Yves

[andre van atten]

Yes, indeed

Andre

Op 13 feb. 2012 22:45 schreef "yvecai" <yve...@gmail.com> het volgende: