sorting transactions

[nug get]

Is there a feature that would allow me to sort the transactions in my files, for example by date, or specific accounts?

Thanks!

Background:
I migrated my 3 years of Gnucash records to beancount. It's all one long list of transactions sorted by date. I'd like to separate my cash and bank account transactions into two different files.

[Martin Michlmayr]

* nug get <nugget....@gmail.com> [2019-12-20 03:58]:

> I migrated my 3 years of Gnucash records to beancount. It's all one
> long list of transactions sorted by date. I'd like to separate my
> cash and bank account transactions into two different files.

You could try:

ledger -f all.ledger print --raw account1 > account1.ledger
ledger -f all.ledger print --raw account2 > account2.ledger

However, ledger print loses some information, so depending on your
ledger file this approach may or may not work.

--
Martin Michlmayr
https://www.cyrius.com/

[Martin Blais]

You will want to write a little python script.

I would operate on source code, not on the data structures, so you keep the original formatting and metadata and comments

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/4b6c37cd-dab7-4683-80fc-e6affac7e1ca%40googlegroups.com.

[nug get]

thanks, a python script would probably do it in the end.

is there a beancount function that would return all objects in my ledger file as list or something? Writing that myself would be beyond my current capabilities, or of irresponsible effort timewise :)

On Friday, 20 December 2019 13:57:54 UTC+1, Martin Blais wrote:

You will want to write a little python script.

I would operate on source code, not on the data structures, so you keep the original formatting and metadata and comments

On Fri, Dec 20, 2019, 06:59 nug get <nugget....@gmail.com> wrote:

Is there a feature that would allow me to sort the transactions in my files, for example by date, or specific accounts?

Thanks!

Background:
I migrated my 3 years of Gnucash records to beancount. It's all one long list of transactions sorted by date. I'd like to separate my cash and bank account transactions into two different files.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bean...@googlegroups.com.

[robm...@google.com]

I wrote a (albeit-hacky) python script to do this for my ledger and made it public here: https://github.com/robmargo/beancount_tools/blob/master/sort_transactions.py

It ignores all lines before a line that has the text "TRANSACTIONS" (everything above that line is account openings/metadata that I dont want to sort). If you need help tweaking this for your needs, let me know.

[Martin Blais]

On Fri, Dec 20, 2019 at 9:30 AM nug get <nugget....@gmail.com> wrote:
>
> thanks, a python script would probably do it in the end.
> is there a beancount function that would return all objects in my ledger file as list or something? Writing that myself would be beyond my current capabilities, or of irresponsible effort timewise :)

That would not work. Beancount transforms your input transactions to
something slightly different (there's more going that it might appear,
unfortunately) and so while round-trips betwee text->python->text are
possible (and tested out), the text you'd get is not quite like the
input you provided. You want to keep your terse inputs. Operate on the
input text itself.

> On Friday, 20 December 2019 13:57:54 UTC+1, Martin Blais wrote:
>>
>> You will want to write a little python script.
>> I would operate on source code, not on the data structures, so you keep the original formatting and metadata and comments
>>
>> On Fri, Dec 20, 2019, 06:59 nug get <nugget....@gmail.com> wrote:
>>>
>>> Is there a feature that would allow me to sort the transactions in my files, for example by date, or specific accounts?
>>> Thanks!
>>>
>>> Background:
>>> I migrated my 3 years of Gnucash records to beancount. It's all one long list of transactions sorted by date. I'd like to separate my cash and bank account transactions into two different files.
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups "Beancount" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an email to bean...@googlegroups.com.
>>> To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/4b6c37cd-dab7-4683-80fc-e6affac7e1ca%40googlegroups.com.
>
> --
> You received this message because you are subscribed to the Google Groups "Beancount" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/a728b410-6f4d-4623-b7fb-e40d03eb0cf8%40googlegroups.com.

[Manuel Amador (Rudd-O)]

Here is some code to do mostly that.  Use is to run it with Python 3, first argument is source file, second argument is output file for matched transactions, third is output file for non-matched transactions, and the fourth and so on are text strings that, if found, will add to the matched, and if not found, will add to the non-matched output.

#!/usr/bin/python3

import sys

matchers = sys.argv[4:]
infile = open(sys.argv[1], "r")
chunk = None
lastone = None

def match(text):
    return all (m in text for m in matchers)


def start_condition(line):
    return line.startswith("20") or line.startswith(";")

def end_condition(line):
    return not bool(line.rstrip("\n"))


with open(sys.argv[2], "w") as matched:
    with open(sys.argv[3], "w") as notmatched:
        for line in infile:
            if chunk is not None:
                chunk += line
                if end_condition(line):
                    # Chunk over.  We print.
                    lastone = (matched if match(chunk) else notmatched)
                    lastone.write(chunk)
                    chunk = None
            else:
                if start_condition(line):
                    chunk = line
                else:
                    matched.write(line)
                    notmatched.write(line)

        if chunk is not None:
            (matched if match(chunk) else notmatched).write(chunk)

To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/a728b410-6f4d-4623-b7fb-e40d03eb0cf8%40googlegroups.com.

-- 
Rudd-O
    http://rudd-o.com/

[Manuel Amador (Rudd-O)]

Actually, just pushed it to a repo.  It's better:

https://github.com/Rudd-O/bean-chunk

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/4b6c37cd-dab7-4683-80fc-e6affac7e1ca%40googlegroups.com.

-- 
Rudd-O
    http://rudd-o.com/

[Martin Blais]

Hmm. It would be nice if the metadata produced by the parser contained both the starting and end line.

This way you wouldn't have to do any parsing to extract the original text.

I'll put that on the list for v3 changes.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/d117b6cb-67e1-3e59-1c39-928cee0672a7%40rudd-o.com.

[Manuel Amador (Rudd-O)]

On 25/11/2020 06.09, Martin Blais wrote:

Hmm. It would be nice if the metadata produced by the parser contained both the starting and end line.

This way you wouldn't have to do any parsing to extract the original text.

I'll put that on the list for v3 changes.

Thanks!

-- 
Rudd-O
    http://rudd-o.com/

[Casey Link]

Many thanks for this script!

Some years ago when I started with beancount I decided to use one beancount file, but separate the accounts by Org headings. This worked for awhile, but now with thousands and thousands of lines it can be confusing to work with. It's much nicer to have everything sorted chronologically. Your script worked perfectly help me reorganize my beancount file.

Thanks again!

Best,

Casey

[polarmutex]

I have been working on a tree-sitter parser for beancount, the node tree produced has the starting / ending line/column of the parsed file. 

https://tree-sitter.github.io/

https://github.com/bryall/tree-sitter-beancount

with that info you should be able to sort the ledger file

I still think my parser has some edge cases that may not work, bu t I see no errors in my files so far

I am writing a beancount language server currently, I may be able to add a code action to do this sorting  in the LSP in the future

[Martin Blais]

Nice. Thank you for sharing.

Are you able to customize what data structures it fills?

In v3 there's a schema for protos, that can be compiled to many languages.

https://github.com/beancount/beancount/tree/master/beancount/ccore/data.proto

Also, with protos you can use gRPC to build a server that accepts the same schema.

Just some ideas,

To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/4396ceeb-4d8c-49d1-9ccb-1acc4927c896n%40googlegroups.com.

[polarmutex]

It generates an AST (abstract syntax tree) that you can query over and use to manipulate the source file,  the developer writes a grammar that specifies the nodes names and structure, so I could change my structure to more align with the internal structure of beancount

I am using it to use with Neovim to try to add as many features to better help use beancount with that editor.  Right now I have coded a plugin to launch a  fuzzy finder window to search over transactions in the current file and then be able to copy that transaction to the current cursor position.  

I am planning on integrating it with my LSP to provide support of the LSP functions.

I am very interested in integrating the c++ libraries once they are in a good state.