Updates: grano QL, granoloader, common schema
13 views
Skip to first unread message
Friedrich Lindenberg
Jul 5, 2014, 11:45:41 AM7/5/14
to gran...@googlegroups.com, Rizmari Versfeld
Hi all,
as we’re doing a sprint on a South African instance of grano at the moment, a lot of good things are happening with the platform. Here’s a quick update on the major improvements that are coming up:
## Grano QL
grano QL is a new module (https://github.com/granoproject/grano-ql) that aims to implement a powerful query-by-example interface for grano. The language is inspired by Freebase’s Metaweb Query Language, and I hope that it can prove a powerful base for future visualisations and browsing interfaces.
The module is currently in a pre-alpha stage, but we’re already able to run complex, nested queries (e.g. "all entities which have a financial link to a company which has a them as a board member”). Hacking this is a lot of fun, but I’ve never actually done a language parser before - so any help is really welcome!
## Granoloader
granoloader (https://github.com/granoproject/granoloader) is a tool that I hope will streamline the ETL process for any grano instances that use bulk data. The API-based command line script can load data into a project using a CSV file and a mapping specification. This means that all data will be loaded from flat files in a way that is explicit and documented. Because granoloader is multi-threaded, it’s also a lot faster than most existing loader scripts.
## Common schema
A recurring theme in discussions about grano is the need for a common base schema that would be available in all projects. I want to do this with an eye towards existing standards (such as Popolo), and based on a concrete example (connectedAfrica in South Africa).
This would really benefit from peer review, so please contribute your ideas on the connectedAfrica wiki and I’ll ping this list again once we’ve got a prototype spec:
https://github.com/CodeForAfrica/connectedAFRICA/wiki/Data-Schema
That all for now, hope everybody is well!
Cheers,
- Friedrich
as we’re doing a sprint on a South African instance of grano at the moment, a lot of good things are happening with the platform. Here’s a quick update on the major improvements that are coming up:
## Grano QL
grano QL is a new module (https://github.com/granoproject/grano-ql) that aims to implement a powerful query-by-example interface for grano. The language is inspired by Freebase’s Metaweb Query Language, and I hope that it can prove a powerful base for future visualisations and browsing interfaces.
The module is currently in a pre-alpha stage, but we’re already able to run complex, nested queries (e.g. "all entities which have a financial link to a company which has a them as a board member”). Hacking this is a lot of fun, but I’ve never actually done a language parser before - so any help is really welcome!
## Granoloader
granoloader (https://github.com/granoproject/granoloader) is a tool that I hope will streamline the ETL process for any grano instances that use bulk data. The API-based command line script can load data into a project using a CSV file and a mapping specification. This means that all data will be loaded from flat files in a way that is explicit and documented. Because granoloader is multi-threaded, it’s also a lot faster than most existing loader scripts.
## Common schema
A recurring theme in discussions about grano is the need for a common base schema that would be available in all projects. I want to do this with an eye towards existing standards (such as Popolo), and based on a concrete example (connectedAfrica in South Africa).
This would really benefit from peer review, so please contribute your ideas on the connectedAfrica wiki and I’ll ping this list again once we’ve got a prototype spec:
https://github.com/CodeForAfrica/connectedAFRICA/wiki/Data-Schema
That all for now, hope everybody is well!
Cheers,
- Friedrich
Reply all
Reply to author
Forward
0 new messages