Lines are not straight in VTM

[Jonas W]

I convert an indoor map from .osm to .map format using osmosis. When i display the map in my Android App the room lines are not straight, although they are straight in the .osm file. Does someone know why this happens?

My rendertheme looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<rendertheme xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" map-background="#fffcfa"
    version="1" xmlns="http://opensciencemap.org/rendertheme"
    xsi:schemaLocation="http://opensciencemap.org/rendertheme https://raw.githubusercontent.com/mapsforge/vtm/master/resources/rendertheme.xsd">

   

    <m closed="yes" k="room">
        <!-- <m v="fence|wall|city_wall" zoom-min="15"> <line
          stroke="#909090" width="0.1" cap="butt" /> </m> -->

        <caption style="bold" fill="#000000" k="name" size="13" stroke="#ffffff" />

        <area fill="#dbdbc9" stroke="#707070" stroke-width="1.0" />

    </m>
</rendertheme>

[Emux]

Can you post small samples of input .osm and output .map to check them?

Are there any screenshots available?

--
Emux

[Emux]

(sorry just saw the attachment in email) :)

There is a PR #226 with the idea of changing some floats to doubles to achieve that.

But @hjanetzek (VTM author) recommended there a far less obtrusive way and due to lack of samples for testing it didn't proceed.

--
Emux

[Longri, Andre Höpfner]

With zoom in to 2m?

--
You received this message because you are subscribed to the Google Groups "mapsforge-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapsforge-dev+unsubscribe@googlegroups.com.
To post to this group, send email to mapsfo...@googlegroups.com.
Visit this group at https://groups.google.com/group/mapsforge-dev.
To view this discussion on the web visit https://groups.google.com/d/msgid/mapsforge-dev/8dc3260f-6c7e-e912-2d11-87f22e9aa1b2%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Emux]

On 31/03/2017 07:55 μμ, 'Longri, Andre Höpfner' via mapsforge-dev wrote:

With zoom in to 2m?

@Longri the question is to whom?

--
Emux

[Longri, Andre Höpfner]

Sorry, to Jonas 

--

You received this message because you are subscribed to the Google Groups "mapsforge-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapsforge-dev+unsubscribe@googlegroups.com.
To post to this group, send email to mapsfo...@googlegroups.com.
Visit this group at https://groups.google.com/group/mapsforge-dev.

To view this discussion on the web visit https://groups.google.com/d/msgid/mapsforge-dev/767e1666-ba42-eea0-03a5-11cdf9abb528%40gmail.com.

[Emux]

In such very large zooms there may be a precision issue.

And 1st candidate for checking would be MapDatabase where it parses the map file.

Also it's worth checking any existing sample also with Mapsforge for comparison.

--
Emux

[Longri, Andre Höpfner]

LatLong are clamp to accurately of 1E6.

That's ~1.5m, or?

[Emux]

On 31/03/2017 08:10 μμ, 'Longri, Andre Höpfner' via mapsforge-dev wrote:

LatLong are clamp to accurately of 1E6.

That's ~1.5m, or?

Do you mean the VTM GeoPoint class?
(LatLong is a Mapsforge class)

Checking the Mapsforge binary map file format specification:
https://github.com/mapsforge/mapsforge/blob/master/docs/Specification-Binary-Map-File.md

"All latitude and longitude coordinates are stored in microdegrees (degrees × 10⁶)"

--
Emux

[Longri, Andre Höpfner]

Whether LatLong or GeoPoint, the result is the same. There is no greater accuracy in the map file.

--

You received this message because you are subscribed to the Google Groups "mapsforge-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapsforge-dev+unsubscribe@googlegroups.com.
To post to this group, send email to mapsfo...@googlegroups.com.
Visit this group at https://groups.google.com/group/mapsforge-dev.

To view this discussion on the web visit https://groups.google.com/d/msgid/mapsforge-dev/dc7ca990-a1c4-604d-3098-8e816bb85aed%40gmail.com.

[Emux]

I'd still like to have some indoor samples (input + output) to test,
as with regular Mapsforge maps the issue is not apparent.

--
Emux

[Jonas W]

@Andre: yes, I zoom in to >2m, because I want to display indoor maps.

@Emux: see attached .osm and .map file. I create the .osm with JOSM, then i convert the .osm to the new format using:

osmconvert64.exe --fake-author --statistics testmap.osm -o=testmap_new.osm -v=2

Then I convert the .osm to map using:

bin\osmosis --rx file=data\testmap_new.osm --mapfile-writer file=testmap_new.map bbox=52.5259193,13.3135157,52.5265404,13.3144585 tag-conf-file=data\tags.xml 

Is there a way to make the lines straight? Otherwise I have to use another tool to display the map.

Greetings and thanks for your help!

Jonas

[Emux]

Thanks for the samples, if there is also a VTM theme working with them, please attach it.

BTW I tested with Mapsforge reader and the result seems the same?
(see screenshot)

Also I see the converted .osm has less precision in coordinates?
e.g. 52.52628585753 -> 52.5262858

--
Emux

[Jonas W]

Hi Emux,

I posted the rendertheme in the first post.

Great finding that osmconvert cuts the last 2 digits! I posted a question on stackoverflow regarding this.

Or do you know why osmconvert cuts the last 2 digits?

Greetings,

Jonas

[Jonas W]

Hi Emux,

as it has been answered on stackoverflow, cutting the last 2 digits from lat/lon should not lead to imprecise maps, as they are in the millimeter region.

So the reason for the lines should be somewhere else. Can you, or someone else think of another reason why the lines are not straight?

Greetings,

Jonas

[Emux]

Have not debugged it further, but important also is how they're stored, see my above note for Mapsforge map file specs.

--
Emux

[Jonas W]

Hi Emux,

in the link that you posted I read that 

"All latitude and longitude coordinates are stored in microdegrees (degrees × 106)."

Does this mean that the issue comes from the dataformat of mapfile? That lat/lon is not stored precise enough?

Greetings,

Jonas

[Emux]

Map writer stores the coordinates in microdegrees (degrees × 1E6) INT.

e.g. GeoUtils.toCoordinateList converting all JTS geometries performs the following:
(int) (coordinate * 1000000.0)

--
Emux

[Jonas W]

Hi Emux,

so this means that the map Write is not suitable for indoor maps, right?

Are there any other tools to create a .map file that are more precise?

Greetings,

Jonas

[Emux]

Mapsforge .map file format was specified that way, so writer and readers work that way.
i.e. the tools follow the format specification

We could need a new map format to publish something different, and requests are not just for precision, but also for flexible tagging, etc.

There was a long discussion some time ago for a new Mapsforge map format, but eventually ran out of steam..

Or could use another format, like vector tiles (OpenScienceMap .vtm or Mapbox .mvt or GeoJSON or TopoJSON) in a DB structure, don't know how size would be though.

--
Emux

[Ludwig]

Where are your "straight" lines? Technically there are no straight lines on a sphere, so if you get to close the poles apparently straight lines will not appear straight.

Joke aside, could this be a projection issue?

--

You received this message because you are subscribed to the Google Groups "mapsforge-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapsforge-dev+unsubscribe@googlegroups.com.
To post to this group, send email to mapsfo...@googlegroups.com.
Visit this group at https://groups.google.com/group/mapsforge-dev.

To view this discussion on the web visit https://groups.google.com/d/msgid/mapsforge-dev/a9571d6f-9078-2bf5-49e0-a7d8475b6605%40gmail.com.

[Emux]

On 10/04/2017 07:45 μμ, Ludwig wrote:

Joke aside, could this be a projection issue?

Had not debugged it deeply, but same rendering appears in both Mapsforge and VTM w/ OpenGL viewers.

Considering that coordinates in map file are stored with fixed microdegrees precision, indoor ways could be affected.

--
Emux

[Emux]

Forgot mentioning that Hannes recommended reviewing VTM MapDatabase parser, could use some work (as based on Mapsforge older implementation).

It's in my long list of library future improvements. :)

--
Emux

[Jonas W]

Hi Emux,

so I would like to try another map format, although it seems much more complicated (especially using Geo-/TopoJSON) when looking at the example sourcecode. I'd like to keep my custom theme for the openstreetmap format that I use, is that possible with vtm map files? And how do I create those? Sorry couldn't find the solution as all results point to the library itself..

Greetings and thanks!

Jonas 

[Emux]

See this topic for a discussion regarding OpenScienceMap .vtm vector tiles.
None has contributed a workable creation guide yet...

VTM default themes play with Mapsforge maps and .vtm tiles, as they use the same tags.

Regarding the other 3 vector tile formats (Mapbox .mvt, GeoJSON, TopoJSON), Mapzen is one of their providers.
But they make changes in OSM tags in the tiles, so a different theme is needed (i.e. our mapzen.xml).
Their creation could be achieved using e.g. the tools in Mapzen repositories.
Currently we have tile decoder in VTM for the .mvt tiles.

Storing offline the tiles can be achieved e.g. in SQLite see TileCache.

--
Emux

[Jonas W]

In order to make it vtm work for my indoor maps, I'd like to change the mapwriter/reader. 

As we have seen, microdegrees in int precision are not precise enough for indoor maps. So i'd like to introduce nanodegrees in long format. My naive approach would be to refactor int microdegrees to long nanodegrees. Will this work or does it brake something that i don't think of? (e.g. something could break because the map file format specification is changed by using long instead of int)

Thank you for your help!

[Emux]

Changing map file internals effectively breaks compatibility as could be a different new format then.

Also don't know if it's the best solution or not use one of the more recent aforementioned formats.

--
Emux

[Ludwig]

I also think that changing the precision will not work (plus is not so easy, because you will need to change all the calculations to compress numbers as is done in the map file format)

I suggest trying out a different approach: you can work with less bits if you assume that everything is within a certain bounding box and the values for this bounding box are added whenever you are retrieving a location. 

Just an example: let's assume your location is 44.3676778 22.3389264. Now if your area of interest is all within 44-45 and 22-23 you do not need to encode the 44 and 22, essentially gaining two digits of precision at the end. 

Not sure if this is clear. I am happy to discuss this further if this is totally unclear. I must also admit that I have never tried this, so the concrete implications are a bit unclear to me as well. 

Ludwig

--

You received this message because you are subscribed to the Google Groups "mapsforge-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapsforge-dev+unsubscribe@googlegroups.com.
To post to this group, send email to mapsfo...@googlegroups.com.
Visit this group at https://groups.google.com/group/mapsforge-dev.

To view this discussion on the web visit https://groups.google.com/d/msgid/mapsforge-dev/404c0b50-9358-3c36-8ec8-aa8cab5dc020%40gmail.com.

[Emux]

> plus is not so easy, because you will need to change all the calculations to compress numbers as is done in the map file format

Indeed the coordinates are stored with single & double delta encoding for size efficiency.

--
Emux

[Jonas W]

Hi Ludwig,

I also thought about this (hacky) solution. Where would I need to substract/add the digits? Only in the conversion microdegreesToDegrees/degreesToMicrodegrees functions?

Greetings,

Jonas

[Jonas W]

I now think this is happening because of the delta encoding.. This delta has to be more precise. Am I right Emux, Ludwig?

Is it a hard task to make this delta more precise? And is there any chance that in the foreseeable future that you or someone else implements this? ;) Or how can I make this a bigger priority?

Greetings,
Jonas

[Emux]

The map file concept is working with stored microdegrees internally, so probably needs a remodeling(?) in order for anything like that to work.

--
Emux

[Jonas W]

Why is it using 10^6(microdegrees) anyways? int can also store 10^7?

[Ludwig]

On 30 May 2017 at 15:08, Jonas W <jonaswi...@gmail.com> wrote:

Why is it using 10^6(microdegrees) anyways? int can also store 10^7?

That seems to be a valid point. I think the current microdegree storage concept has its origins in the early desire by mapsforge to be a Google maps replacement. The largest positive/negative values we need to store are +-180, so unless I am mistaken 180*10**7 < 0x7ffffff. 

So a E7 conversion factor would have been fine. At this point I am not sure if anything else forced the decision to use a 10**6 conversion factor. 

In a general sense, this choice is unlikely to change as simply too many maps are out there that would be invalidated. 

But with a quick browse through the writer code, there seems nothing that would not allow 10**7 latlongs to be included in the map, the calculations all seem to be in integers. (There must be somewhere where the conversion takes place when reading the OSM data).

On the reader side, if we did everything correctly, simply changing the CONVERSION_RATIO to 10**7 should do the trick. 

Obviously, I do not believe it will be so simple.

However, looking at your example data, having an order of magnitude more will not solve your problem of jagged lines. 

The problem, in a nutshell, is that you have a line x1y1 ---- x2y2 where x1 != x2 and y1 !=y2 (meaning your line is neither horizontal nor vertical) Now with your complex geometry you have additional points on this line (to mark doors etc) The problem is that you can choose your x3 arbitrarily (within the range of precision), but with a given precision most of the y points will not lie exactly on the required line. 

I currently have no good idea how to fix this (obviously some map programs solve this problem), apart from totally changing the implied scale of the map as I pointed out in a previous comment. 

Ludwig

[Emux]

Google Maps Android v1 has microdegrees, while from Google Maps Android v2 doubles were introduced.

Early projects tried to be similar, e.g. Osmdroid before & after.
And when Mapbox forked Osmdroid, they moved it to doubles, then of course opted for OpenGL.

Mapsforge is a bit different, it uses doubles for all its structure / overlays and user API, except the map storage.
Why the specific precision? Because the needs were for map detail up to buildings level where indoor cases weren't considered and standard 1E6 was suitable.

Like Ludwig mentioned, backwards compatibility is always important, though improvements should be encouraged.
For example 2 years ago we introduced multilingual Mapsforge maps (spec v4) and only recently they started to gain traction.

Other formats were built from the start with far greater precision (32 bit varint).
e.g. VTM vector OSciM .vtm or Mapbox .mvt tiles

--
Emux

[Emux]

I'd suggest a different approach:

See JeoIndoorMapActivity example in VTM, which uses GeoJSON data for indoor mapping.

--
Emux

[Jonas W]

geoJSON is much bigger than the .map Format & it seems the rendering is slower. Also styling is more difficult/buggy.

Where would I request an enhancement to the .map format to use e.g. nanodegrees instead of microdegrees?

[Emux]

GeoJSON is a well known format with the benefit of much simpler parsing / rendering as proved in VTM tile decoder for Mapzen MVT vs GeoJSON vector tiles.

Regarding Mapsforge files, modifying the core of an established fixed binary format is not a trivial / fast process. Considering the fact that our primary concern is always to not break the library.

So if having something to test I'd be happy to review. :)

BTW recently Mapsforge format advanced to v5 without breaking backwards compatibility (as it should) - for supporting variable tag values via wildcards, e.g. introducing S3DB extrusion tags.

--
Emux

[Emux]

> Also styling is more difficult/buggy.

That does not mean that if GeoJSON works for our purpose but its renderer seems with issues, we abandon so easily that format and choose to "rewrite" Mapsforge format just because it "seems" having nicer renderer.

If you're referring to CartoCSS that's not used in VTM themes, but in some external GeoJSON overlays - so it's a completely different case, i.e. don't expect the vtm-jeo functionality to play with Mapsforge files.

After all the same map / theme engine & rules work exactly the same with Mapsforge files, MVT + GeoJSON vector tiles.

So maybe it's wiser to improve the working format's renderer instead, that would lead to much faster / improved results with probably less work.

--
Emux