Visidata

219 views
Skip to first unread message

Martin Blais

unread,
Jul 25, 2020, 3:51:46 PM7/25/20
to Beancount
Hi,
I discovered a pretty awesome tool yesterday related to text files /  tables / text processing in the console, I thought I'd highlight it here since I suspect a lot of you might be interested in this.

I've had it in the back of my mind to build a renderer / visualizer for CSV files for curses, either as an Emacs mode or as a standalone tool, for a very long time. I remember looking for this years ago and it didn't exist and I thought "this is crazy." And it seems as the years go by, I'm using less and less GUI programs--tmux was a big discontinuity in that direction--and basically, I can be pretty complete with just a terminal and Chrome (of course in the terminal I need to run Emacs and bash and all that good stuff). So the idea that I can't even render and filter and arrange tables in the console was bother me.

Well someone has built one, in Python, and it's really slick and looks stable:
It not only renders the CSV files as tables in the console, but provides a bunch of clever editing features as well.

As I'll be moving Beancount's query engine to work on more general tables, this tool will be a really great complement, especially if it can be used as a pager.

Enjoy,

Ben Blount

unread,
Jul 30, 2020, 6:57:08 AM7/30/20
to bean...@googlegroups.com
This looks like a great tool, thanks for sharing! I particularly like the ability to compute a new column via a python snippet.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAK21%2BhNW57ujJ4tu6AUefo_uFZ_woL-r%2BjNiCi6Tb6quTnG2Pw%40mail.gmail.com.

Red S

unread,
Jul 31, 2020, 4:56:28 PM7/31/20
to Beancount
That is indeed a killer feature, as as the diff; and the -f fixed for processing command output.

This really fills a hole in terminal processing, thanks for sharing, Martin.

Speaking of awesome terminal clients, I hope most folks are familiar with tig (git ncurses client)? If not, let me say I switched to it years ago, and it instantly replaced some 90% of my command line git interaction. Includes fully customizable keyboard shortcuts, and arbitrary command mappings.

Also noticed Visidata allows for creating new terminal programs, and one that's been created is vgit, though I haven't tried it yet (tig does everything I need).

TRS-80

unread,
Oct 27, 2020, 2:39:27 AM10/27/20
to bean...@googlegroups.com
> On 2020-07-31 12:56, Red S wrote:
>
> Speaking of awesome terminal clients, I hope most folks are familiar
> with tig (git ncurses client)? If not, let me say I switched to it
> years ago, and it instantly replaced some 90% of my command line git
> interaction

Us Emacs users are blessed with Magit (a git porcelain inside
Emacs). In fact, so many people started getting into Emacs just
to use Magit that I now refer to it as "one of the two gateway
drugs to Emacs" (the other being Orgmode). lol

Cheers!

TRS-80

Martin Blais

unread,
Jan 25, 2021, 9:40:37 PM1/25/21
to Beancount, alim...@googlemail.com
Here's another absolute source of joy: petl

While I like Pandas, I find its various attempts to leverage Python syntax difficult to remember and I always fumble with dataframes and indices.
A while ago I wanted something more predictable, using just regular Python tuples and lambdas using a single chaining syntax.
Not to do any (or much) computation, but rather, just to do CSV file cleanup and normalization.
I came up with this "one-file Pandas" proof-of-concept (warning: not-so-great unfinished code ahead):

The resemblance with petl could not be overstated.
petl is a pleasure to use if you deal with dirty tables (e.g. writing importers).
I'm going to delete my half-assed table.py and start using petl library.
I've been processing options chains all day with it and working with tables has never felt so liberating...

Enjoy,


--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

redst...@gmail.com

unread,
Jan 30, 2021, 2:50:13 AM1/30/21
to Beancount
Excellent find. I've been wanting to refactor my importers to separate the source parser (ofx, csv, txt, etc.) from the transaction constructor (for investments, banking, etc.), which can be somewhat complex and filled with special cases, especially for investments. petl was just what I'd been waiting for. I'll post an an importer framework I've created with it in a minute.

Thanks for sharing!

Ben Blount

unread,
Apr 11, 2021, 9:37:17 PM4/11/21
to Beancount
Did either of you end up making some end to end flows with this that you'd be willing to share?
I'm looking to make a pivot-table of account rollup balances over time and something like PETL would be a great fit.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/eb01f850-dcbb-4f2f-8af0-7585b15a92d6n%40googlegroups.com.

Martin Blais

unread,
Apr 11, 2021, 9:42:55 PM4/11/21
to Beancount
Using it a ton but not with Beancount yet


To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CACGEkZtgioG_77zGnbLhYoqU%3DEnmA2vPWD-cV9OMAoRK9pW3Vw%40mail.gmail.com.

redst...@gmail.com

unread,
Apr 11, 2021, 9:43:08 PM4/11/21
to Beancount
Yes, extensively in my importers.

Ben Blount

unread,
Apr 12, 2021, 7:13:37 AM4/12/21
to Beancount
Thank you both!

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/f2a604fa-5b87-4b8b-9ca6-5138b46b444bn%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages