[ANN] Introducing semantic-csv, a tool for higher-level CSV parsing/processing functionality

[Christopher Small]

Hi everyone

I’m pleased to announce the release of semantic-csv, a humble library for working with CSV data.

Existing Clojure libraries for working with CSV data (clojure.data.csv and clojure-csv being the most notable), only concern themselves with the syntax of CSV; They take CSV text, transform it into a collection of vectors of string values (or when writing, write from a sequence of string vectors), and that’s it. Semantic CSV takes the next step by giving you tools for addressing the semantics of your data, helping you put it into the form that better reflects what it means, and make it easier to work with.

Features

• Absorb header row as a vector of column names, and return remaining rows as maps of column-name -> row-val
• Write from a collection of maps, given a header
• Apply casting/formatting functions by column name, while reading or writing
• Remove commented out lines (by default, those starting with #)
• Compatible with any CSV parsing library returning/writing a sequence of row vectors
• (SOON) A “sniffer” that reads in N lines, and uses them to guess column types

Structure

Semantic CSV is structured around a number of composable processing functions for transforming data as it comes out of or goes into a CSV file. This leaves room for you to use whatever parsing/formatting tools you like, reflecting a nice decoupling of grammar and semantics. However, a couple of convenience functions are also provided which wrap these individual steps in an opinionated but customizable manner, helping you move quickly while prototyping or working at the REPL.

Where you come in

Semantic CSV is still in alpha, but I’m excited to start getting people using it and providing feedback. I’m particularly interested at this phase in hearing what people think of the overall structure, what pain points come up, and what features would be nice. And bugs of course. Feel free to submit feedback via Github issues or the project chat room.

https://github.com/metasoarous/semantic-csv

Thanks for your time; I hope you find this useful.

Chris Small

​

[Sébastien Orban]

Wow, nearly perfect timing for me - lot of csv to parse, and transform into their final destination (mostly map with correctly asserted/cast value - some with instaparse, joy of badly separated data). So, cast-with is very appreciated !

Seem to cover everything - even lazy loading so... what's not to like ?

[Christopher Small]

Fabulous. I'm glad you'll find it useful :-)

Chris

   

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/bmxEwImkfTQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jony Hudson]

This is great - just what I need!

Jony

[Christopher Small]

Glad you like it Jony :-) And thanks for your contributions.

There are a few things we'll be working on before putting out the first non-alpha release. In particular, better casting error handling options, some safer casting function helpers, improved testing coverage (already well underway, thanks to @ballPointPenguin who's been helping out), and a couple more transformation functions. Once that's ready, I'll send out another announcement.

Thanks again.

Chris