Understanding MyLocal Org Hierarchy and Data Types

[Vibhatha Abeykoon]

Hello everyone,

We have been analyzing the MyLocal dataset as part of our effort to port application data into databases to develop a scalable solution for the existing backend.

One of the initial steps in this migration process is to understand the hierarchy of Sri Lanka’s governing entities in a parent-child structure for the following categories:

Province
District
ED
PD
GND
LG
MOH
DSD

Upon examining the mylocal_services dataset, we noticed that the subs and supers relationships are somewhat complex. Some entities within the same category have different supers, even when they belong to the same governing body. For instance, one governing entity in a given category may have MOH as its parent, while another entity of the same category may have a different parent entity.

To streamline queries and enhance interpretation, we plan to structure this data in a graph database. This will allow us to efficiently determine relationships—such as identifying which governing body a DSD reports to.

Additionally, we would like to clarify the settings and how the sends are used for GeoJSON and TopoJSON data.

@Nuwan Senaratne, could you provide some insights on the topics discussed here?

Looking forward to your input.

Best, 

Vibhatha

[Nuwan Senaratna]

Regions

It's a little complicated. 

The logical hierarchies are as follows:

* Country -> Province -> District -> DSD -> GND

* Province -> ED -> PD

* Province -> District -> LG 

* Province -> MOH

(where X -> Y, means many Ys are contained in exactly one X. E.g. Western Province -> Colombo Electoral District -> Borella Polling Division)

I only had exact geo boundaries for  Country -> Province -> District -> DSD -> GND. And so to map the other regions, I approximated each as a collection of GNDs. In other words, I made GNDs the "universal child region". In some cases, these are accurate. For example, the Kuppiyawatta West GND is wholly contained within the Borella PD. But in some cases, GNDs split across multiple "parent" regions. This isn't a huge problem because GNDs are typically quite small. 

We could get the exact geo polygons from the survey department for all the region types - but I didn't attempt that. 

Related, I wrote this article a while ago to explain some of the above - https://medium.com/on-economics/where-does-the-president-live-971dd541082c

"supers" and "subs" are inferred variables. X is a super of Y if all of Y's GNDs are contained within X. Y is a sub of X, if X is a super of Y. I believe I removed this variable, as it wasn't very useful. But old versions of the code/data might have it. 

GeoJSON and TopoJSON data

GeoJSON stores regions as polygons. For example, Ratnapura District will be one giant polygon. 

TopoJSON tries to achieve better storage by representing the polygons as arcs. For example, Ratnapura and Galle Districts share a boundary, and so this shared arc (or could be multiple arcs) are stored only once. Because of this more complicated representation rendering is (slightly) more complicated. 

This is the most important difference. There are a few others. 

See https://en.wikipedia.org/wiki/GeoJSON for a good discussion on both file types.   

...

Thanks!

-Nuwan

www.nuwan.org · LinkedIn · Medium · @nuuuwan (X/Twitter) · GitHub

This too shall pass

--
You received this message because you are subscribed to the Google Groups "LDF Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev+uns...@datafoundation.lk.
To view this discussion visit https://groups.google.com/a/datafoundation.lk/d/msgid/dev/abde8fe0-a663-4fc3-bcb2-3fa15bf032b0n%40datafoundation.lk.

[Vibhatha Abeykoon]

Nuwan thank you for this wonderful explanation, I think I got a better idea about the data structures and their relationships. I am looking for the most efficient storage formats for each of these data types. 

To view this discussion visit https://groups.google.com/a/datafoundation.lk/d/msgid/dev/CAOWTO4G522vNEk8CLgQO1zAchLVCB_ARkpmtq5Mt%3DSUh308sRQ%40mail.gmail.com.