Getting started with factdb and dumping the data to other formats

141 views
Skip to first unread message

Jason Rogers

unread,
Jan 21, 2018, 2:58:42 PM1/21/18
to openmundi
Hi, I'm interested in using the data posted in factdb, but I'm quite confused about how to properly use the database and I can't find much in the way of a manual or tutorial.

First, if there is a good manual/tutorial please point me to it and I'll read through it thoroughly. If not, then here are my questions:

1. I see there are separate pages for factdb, openmundi, world.db, etc. What is the relationship between these? Are they different (outdated) versions of the same thing? Or part of a larger parent project, etc.?

2. As someone who is familiar with coding/terminal/scripting, but not familiar with json or working with databases, what is the proper way to use and access the data?

3. Would it be possible to post a dump of all the data to a csv file or similar? Ultimately I want to have a data table where rows are countries and columns are all categories. It would also be ok to have one master json file which I can then parse in R. Alternatively, if you could even just tell me conceptually the right steps to follow to get to this result, I can probably figure out the rest myself.

Thank you for maintaning this project!

Gerald Bauer

unread,
Jan 21, 2018, 3:13:42 PM1/21/18
to Jason Rogers, openmundi
Hello,

Thanks for your interest in the factbook. Please note I do NOT work
for the CIA - I'm just a volunteer.

You can find "dumps" of all factbook data in json at the
/factbook.json repo - see
https://github.com/opendatajson/factbook.json

There's also a guide on how to import the datasets into MongoDb (a
"schemaless" database) - see
https://github.com/opendatajson/factbook.json/blob/master/MONGO.md


The factbook data is not really all 100% tables and columns it's a
mix of table-oriented and document-oriented. So importing in SQL or
converting to CSV only works for a subset and that subset you can
download from the official CIA site :-)

Look for the alternative factbook project by Ian Coleman that tries
to clean-up / unify the datasets so you can easier import into tables
/ convert to comma-separated values (csv) records -
https://github.com/iancoleman/cia_world_factbook_api

Greetings from Vienna. Let us know how it goes. All the best. Cheers.
> --
> You received this message because you are subscribed to the Google Groups
> "openmundi" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to openmundi+...@googlegroups.com.
> To post to this group, send email to open...@googlegroups.com.
> Visit this group at https://groups.google.com/group/openmundi.
> For more options, visit https://groups.google.com/d/optout.

Gerald Bauer

unread,
Jan 21, 2018, 3:26:56 PM1/21/18
to Jason Rogers, openmundi
Hello,

One more thing and for more detail why it might not be so easy to
convert the data (straight) to tables and columns, see the issue #201
/ discusson at the datasets/registry repo titled "CIA World Factbook"
-> https://github.com/datasets/registry/issues/201 includes comments
form Ian Coleman too.

Cheers.
Reply all
Reply to author
Forward
0 new messages