CSV-stat: JSON-stat for the people

[Xavier Badosa]

JSON-stat stores data in a flat array. This has two main benefits: it's general (all data structures end up taking the same simple form) and non-redundant (=small). Which makes JSON-stat perfect for APIs (transport-efficient and universal). But using a flat array has its costs: for the consumer, data in a JSON-stat document is not human-readable (it's cryptic); for a producer, creating a JSON-stat document is not a technically trivial task (the producer needs to master the flattening process).

That's why having a more readable format based on existing standards (=processable with existing tools) but semantically as rich as JSON-stat seems a good idea.

Welcome to CSV-stat: https://github.com/badosa/CSV-stat

Because CSV-stat is regular CSV, it can be processed with CSV tools (for example, it can be read and written using a spreadsheet). Because data is expressed in a tabular form, with records and columns (and not in a flat array), it's human readable and easy to produce. Because it's richer than regular CSV, CSV-stat/JSON-stat can be converted back and forth.

Some CSV-stat interesting features:

• Single document in a single syntax

CSV-stat is not a container with different files using different syntaxes (vs. the Tabular Data Package).

• Explicit delimiters
  

CSV-stat metadata header contains information on the delimiters used in the document. This means that CSV-stat can be automatically interpreted without looking at its actual contents. This is not the case of regular CSV.

Some CSV-stat use cases:

• Sharing of JSON-stat datasets
  

A JSON-stat user can share data in CSV-stat with a non-JSON-stat user, knowing that no information is lost in the process.

• Revision and editing of JSON-stat datasets

Data can be easily reviewed in CSV-stat. Editing the contents of a dataset is easier in CSV-stat than in JSON-stat: JSON-stat can be converted to CSV-stat, edited and converted back to JSON-stat.

• Production of JSON-stat datasets
  

Creation of CSV-stat from databases and even spreadsheets shouldn't be difficult because CSV-stat expresses data as a table and because there's no need to deal with the JSON-stat flattening process. Then CSV-stat can be automatically converted to JSON-stat.

It is also possible to produce JSON-stat datasets from a regular CSV by adding the necessary extra header lines of CSV-stat metadata and convert the result to JSON-stat.

Tools already available:

From the command line (JSON-stat Command Line Conversion Tools):

$ jsonstat2csv oecd.json oecd.jsv --rich

$ csv2jsonstat oecd.jsv oecd.json

From client JavaScript or Node.js: toCSV() and fromCSV() in the JSON-stat Javascript Utilities Suite.

A sample file:

JSON-stat

https://json-stat.org/samples/galicia.json

CSV-stat

https://json-stat.org/samples/galicia.jsv