Issue #310: sort all entries when extracting multiple files (blais/beancount)

[gyh]

New issue 310: sort all entries when extracting multiple files
https://bitbucket.org/blais/beancount/issues/310/sort-all-entries-when-extracting-multiple

gyh:

I found that when using bean-extract to extract multiple files, the output entries are divided by files and sorted within that file's entries. And then I checked the extract.py and confirmed it.
Is it a little disorderly if just copy all the unsorted entries into an beancount file? Or maybe it is intentionally designed like this because there seem to be no issues discuss it.
However, I'd appreciate all the entries from multiple files can be printed in time sequence, so is it possible to add one choice into bean-extract? Or I can make a PR for this.

[Filip Miletic (Филип Милетић)]

It doesn't seem like it would be useful to everyone.  For example, if you split your transactions into multiple files, having to detangle them by date from bean-extract output would be a frustrating  and unnecessary exercise.

On the other hand, you can cut and paste the unsorted postings into a single file and beancount would work just fine, even if entries themselves are not sorted by time.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20180627162747.18858.25492%40celery-worker-110.ash1.bb-inf.net.
For more options, visit https://groups.google.com/d/optout.

[Jason Chu]

For me I process all files from a single institution at once. Generally those files map directly to different accounts. I split all my transactions by account (within a single file), so this is ideal for me.

A flag could probably do what you want with minimal interruption to other people's workflows.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAKaOXii_ukvaOLdNKmVWgJPai-LxO8b%2BXLdOLHCRM26kptJhGQ%40mail.gmail.com.

[Justus Pendleton]

It seems like having a bean-sort, in addition to bean-format, would help with this kind of thing, while also keeping it in a standalone, reusable app.

I've often wanted to sort my bean file, thoughi acknowledge that is mostly just OCD and I can always get a sorted view via fava and other UIs.

[shreedha...@gmail.com]

Hey

Here's a little python script that essentially does what you want:

$ cat sort-txns.py
#!/usr/bin/env python3
from beancount import loader
from beancount.parser import printer
from beancount.core import data
import sys


entries, errors, options = loader.load_string(sys.stdin.read())


for entry in data.sorted(entries):
    printer.print_entry(entry)

It reads the transactions from stdin.That means I can select the transactions I want to sort in VIM and then do :'<,'>!./sort-txns.py, which replaces the selected text with sorted transactions. Do note that comments are not carried over (since they're essentially ignored by the parser).

Moving this (or an improved version) to an official tool would be helpful though.

[Martin Blais]

You got it:

https://bitbucket.org/blais/beancount/commits/80d30d6896cf5fdcff8c1156cab77107ee8e0f96

--

You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/04d5881e-c1f9-4c32-85c3-88128a6b209d%40googlegroups.com.

[Justus Pendleton]

On Saturday, July 7, 2018 at 6:37:50 PM UTC+7, shreedha...@gmail.com wrote:

It reads the transactions from stdin.That means I can select the transactions I want to sort in VIM and then do :'<,'>!./sort-txns.py, which replaces the selected text with sorted transactions. Do note that comments are not carried over (since they're essentially ignored by the parser).

This is a great first step and useful in many circumstances, so I'm definitely in favor of adding it to a contributed section of the repository as Martin has done.

The reliance on beancount's parser like this means it has some shortcomings, though. Here's a few things I've discovered while testing it out:

- as mentioned, it discards comments

- it also discards balance statements

- it will result in "P" entries being inserted into the file. These are created "under the covers" by beancount when it sees a pad statement but normally users don't see them

- include statements will (naturally) get expanded inline, which probably isn't what the user wants; especially with regard to price statements showing up interleaved in the stream

- option statements will be discarded

- document statements (which the user doesn't normally see) will be added for every document found

- beancount allows the user to leave one leg of a transaction implicit; this will will change that to make it explicit

That is I can write:

2000-01-01 "Eat lunch"

  Assets:WellsFargo -10 USD

  Expenses:Food

and it will get rewritten to

2000-01-01 "Eat lunch"

  Assets:WellsFargo -10 USD

  Expenses:Food 10 USD

- Likewise, inventory will get rewritten to be more explicit. Usually this just means adding a date so that -100 FOO {30 USD} becomes -100 FOO {30 USD, 2018-01-07}

- All commas from numbers will be removed when it is rewritten. So if you buy a house and write 10,400,000,000 it will be rewritten to 10400000000.

Despite all of that, I think that if you keep it to the intended use cases -- selecting transactions and piping it through, as opposed to just throwing an entire file at it -- it has a lot of value and I'll use it myself.

It would be nice to find a way to keep the commas in numbers, though!