readosm: member_type different when reading XML and PBF?

[Geoff Leyland]

Hi,

I’m new here (but I wrote a Lua binding for readosm[1]).

As far as I can tell, readosm uses a different set of values for the member_type field of the readosm_member_struct when it’s reading XML and when it’s reading PBF.

The relevant code for XML is:

if (strcmp (attr[i + 1], "node") == 0)
type = READOSM_MEMBER_NODE;
if (strcmp (attr[i + 1], "way") == 0)
type = READOSM_MEMBER_WAY;
if (strcmp (attr[i + 1], "relation") == 0)
type = READOSM_MEMBER_RELATION;

Where the defines are:

#define READOSM_MEMBER_NODE 7361
#define READOSM_MEMBER_WAY 6731
#define READOSM_MEMBER_RELATION 3671


Whereas for PBF it’s:

int type = *(packed_types.values + i);

which is the value read from the PBF file. I haven’t followed this all the way through, but the relevant protobuf definition is:

message Relation
{
enum MemberType
{
NODE = 0;
WAY = 1;
RELATION = 2;
}
…

And in practice, I get 0 and 1 when reading .pbf files, and 6731 and 7361 when reading xml (I haven’t seen a relation with a relation as a member, but I haven’t looked hard either).

Is this intended behaviour? It took me a while to work out what was going on.


Warm Regards,
Geoff



[1] https://github.com/geoffleyland/lua-readosm

[a.fu...@lqt.it]

Hi Geoff,

you are approaching ReadOSM the wrong way.
the public API is straightforward simple and has just 3 function calls:

READOSM_DECLARE int
readosm_open (const char *path, const void **osm_handle);

READOSM_DECLARE int
readosm_close (const void *osm_handle);

READOSM_DECLARE int
readosm_parse (const void *osm_handle, const void *user_data,
readosm_node_callback node_fnct, readosm_way_callback way_fnct,
readosm_relation_callback relation_fnct);

for more details please see:
http://www.gaia-gis.it/gaia-sins/readosm-1.0.0b-doxy-doc/readosm_8h.html

any other C function you'll find in the sources is clearly labelled
as "static"; this means that they are private internal methods and
are never expected to be directly invoked from outside the library
itself. More precisely, the caller app is strictly forbidden to
directly call such internal methods because they lack any public
link symbol.

You are obviously free to study and review the code as you wish
better, but this doesn't implies that you are expected to extrapolate
single code snippets out of the intended overall scope and context.


----------------------------

the main goal of ReadOSM exactly is the one to support an "abstract"
access to the underlying input file, completely hiding if it actually
is an XML or a ProtoBuf (PBF) file.

the caller application is simply required to perform three actions:

a) opening the OSM file: reasosm_open()
b) registering three callback functions and starting the parser:
readosm_parse()
c) and finally the caller is expected to nicely finalize the OSM
file when the parser terminates: readosm_close()

the OSM parser once started will then internally handle any
format-specific action in a completely transparent way, and it
will automatically invoke the appropriate callback each time
that a NODE, WAY or RELATION is found.
the main task expected from the caller is simply implementing
the callback functions in the most useful way, anyway the caller
is never expected to directly interact with the OSM file if not
thanks to parser intermediation.

here you can find few practical examples about how to correctly
use the ReadOSM library:
http://www.gaia-gis.it/gaia-sins/readosm-1.0.0b-doxy-doc/examples.html

OTH
Sandro

[Geoff Leyland]

Hi Sandro,

Thank you for your reply, and sorry for not explaining the situation well.

On Thursday, 25 June 2015 11:49:10 UTC+12, sandro furieri wrote:

 

you are approaching ReadOSM the wrong way.
the public API is straightforward simple and has just 3 function calls:

I believe I'm using the API the right way, but I'm happy to be corrected!

 

any other C function you'll find in the sources is clearly labelled
as "static"; this means that they are private internal methods and
are never expected to be directly invoked from outside the library
itself. More precisely, the caller app is strictly forbidden to
directly call such internal methods because they lack any public
link symbol.

To be clear, I don't call internal methods.  I noticed something was odd when using the public API, and looked at the code to see if I could work out what was up.

here you can find few practical examples about how to correctly
use the ReadOSM library:
http://www.gaia-gis.it/gaia-sins/readosm-1.0.0b-doxy-doc/examples.html

 

Here's an example using test_osm1 from 1.0.0d.  I've downloaded what I understand is the same file in XML [1] and PBF [2] from geofabrik (a map of Fiji, because it's small).

For the XML file:

test_osm1 fiji-latest.osm | grep member

    <member type="way" ref="330416776" role="inner" />

    <member type="way" ref="22571055" role="outer" />

...

Note that the members have types.  But with a PBF:

test_osm1 fiji-latest.osm.pbf | grep member

        <member ref="330416776" role="inner" />

        <member ref="22571055" role="outer" />

...

These are the same nodes, but they have no types.

So this what I see with the public API, and I apologise that my earlier email went straight into diagnosis without properly explaining the behaviour I see more.

Nonetheless, I think the diagnosis in the earlier email is on the right track.

Warm Regards,

Geoff

[1] http://download.geofabrik.de/australia-oceania/fiji-latest.osm.bz2

[2] http://download.geofabrik.de/australia-oceania/fiji-latest.osm.pbf

[a.fu...@lqt.it]

On Thu, 25 Jun 2015 00:13:32 -0700 (PDT), Geoff Leyland wrote:
>> Here's an example using test_osm1 from 1.0.0d. I've downloaded wha

> d is the same file in XML [1] and PBF [2] from geofabrik (a map of
> Fiji, because it's small).
>
> For the XML file:
>
> test_osm1 fiji-latest.osm | grep member
>

> ...
>
> Note that the members have types. But with a PBF:
>
> test_osm1 fiji-latest.osm.pbf | grep member
>

> ...
>
> These are the same nodes, but they have no types.
>

Hi Geoff,

sorry for any previous misunderstanding; only after reading
your latest e-mail I finally realized that it actually was
a bug report ;-)

All right, the bug is now definitely fixed in readosm-1.0.0e;
you simply have to rebuild the readosm library starting
from the more recent sources.

I've added a cross reference link to your LUA binding in
the ReadOSM project home page.

many thanks for noticing and reporting this annoying bug,
Sandro

[Geoff Leyland]

On Thursday, 25 June 2015 21:14:39 UTC+12, sandro furieri wrote:

many thanks for noticing and reporting this annoying bug,

Awesome.  Thanks for the quick fix!