ledger <--> beancount

[Gour]

Hello, I've installed beancount on my openSUSE Tumbleweed machine in order
whether I could migrate to it from ledger-cli.

I understand there are some syntax differences between the two, but wonder if
there is some way to (easily) convert from one to the other?

I tried loading my current ledger-cli file, but it looks that the problem is
lack of Unicode support in Beancount since I got many Lexer/Syntax errors?

I'm not such a heave ledger-cli user and having web-view along with some
other beancount-goodies could be nice...tried hledger while using Debian, but
since I moved to openSUSE I did not want to fiddle with installing Haskell
deps. Very soon I plan to move to Free/PC-BSD and there are no ports
either for hledger nor for Beancount, but considering I plan to jump into
Python (migrating to Python-powered static-site-generator) and do not
consider learning Haskell in this lifetime, Beancount sounds as better option
- after it gets proper Unicode support - providing there is easy way to
fallback to Ledger if required.

Any hints?


Sincerely,
Gour

--
Those who are on this path are resolute in purpose,
and their aim is one. O beloved child of the Kurus,
the intelligence of those who are irresolute is many-branched.

[Martin Blais]

On Fri, May 22, 2015 at 4:09 AM, Gour <go...@atmarama.net> wrote:

Hello, I've installed beancount on my openSUSE Tumbleweed machine in order
whether I could migrate to it from ledger-cli.

I understand there are some syntax differences between the two, but wonder if
there is some way to (easily) convert from one to the other?

There is an easy way to convert from Beancount to Ledger: bean-report has a command to do that: 

  bean-report <filename> ledger

(There are a few minor warts in this automatic conversion, in particular, Beancount can do things Ledger cannot, like combine postings held at cost and at price in two different currencies, but I will eventually make it output two transactions to fix this. Otherwise I think it works pretty well.)

Converting the other way has to be done manually so far.

It might be worthwhile searching the mailing-list for hints on what others have done.

I once began writing a doc about making conversions but never finished it.

This may also be useful:

http://furius.ca/beancount/doc/comparison

 

I tried loading my current ledger-cli file, but it looks that the problem is
lack of Unicode support in Beancount since I got many Lexer/Syntax errors?

Beancount does not currently support Unicode.

I want to do it someday, but it'll require a lexer rewrite, and it's a low priority item.

I'm not such a heave ledger-cli user and having web-view along with some
other beancount-goodies could be nice...tried hledger while using Debian, but
since I moved to openSUSE I did not want to fiddle with installing Haskell
deps. Very soon I plan to move to Free/PC-BSD and there are no ports
either for hledger nor for Beancount, but considering I plan to jump into
Python (migrating to Python-powered static-site-generator) and do not
consider learning Haskell in this lifetime, Beancount sounds as better option
- after it gets proper Unicode support - providing there is easy way to
fallback to Ledger if required.

Any hints?

Start here:

http://furius.ca/beancount/doc/install

 

Sincerely,
Gour

--
Those who are on this path are resolute in purpose,
and their aim is one. O beloved child of the Kurus,
the intelligence of those who are irresolute is many-branched.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/87fv6pc9tp.fsf%40atmarama.ddns.net.
For more options, visit https://groups.google.com/d/optout.

[Gour]

Martin Blais <bl...@furius.ca> writes:

> There is an easy way to convert from Beancount to Ledger: bean-report has a
> command to do that:
>
> bean-report <filename> ledger

It does not work here, iow. I get:

gour@atmarama ~/financije> bean-report count.ledger ledger
/home/gour/financije/count.ledger:5: syntax error, unexpected ERROR

/home/gour/financije/count.ledger:5: Lexer error; erroneous token: 'Opening'

/home/gour/financije/count.ledger:5: Lexer error; erroneous token: 'Balances'

/home/gour/financije/count.ledger:7: Lexer error; erroneous token: ':eur'

/home/gour/financije/count.ledger:8: Lexer error; erroneous token: ':hrk'

[...]

and the head (line #s added by me) of my count.ledger looks as follows:

#1 ; year 2015
#2
#3 ;include directives.ledger
#4
#5 2014/12/31 * Opening Balances
#6 Assets:ENC 59.48 HRK
#7 Assets:Gotovina:eur 1088.60 EUR
#8 Assets:Gotovina:hrk 1,470.95 HRK
#9 ...

> Converting the other way has to be done manually so far.
> It might be worthwhile searching the mailing-list for hints on what others
> have done.

OK.

> Beancount does not currently support Unicode.
> I want to do it someday, but it'll require a lexer rewrite, and it's a low
> priority item.

Too bad. It seems it's showstopper for me since I annotate my ledger by using
native (Croatian) characters...


Sincerely,
Gour

--
When your intelligence has passed out of the dense forest
of delusion, you shall become indifferent to all that has
been heard and all that is to be heard.

[Martin Blais]

On Fri, May 22, 2015 at 8:26 AM, Gour <go...@atmarama.net> wrote:

Martin Blais <bl...@furius.ca> writes:

> There is an easy way to convert from Beancount to Ledger: bean-report has a
> command to do that:
>
>   bean-report <filename> ledger

It does not work here, iow. I get:

gour@atmarama ~/financije> bean-report count.ledger ledger
/home/gour/financije/count.ledger:5:       syntax error, unexpected ERROR

/home/gour/financije/count.ledger:5:       Lexer error; erroneous token: 'Opening'

/home/gour/financije/count.ledger:5:       Lexer error; erroneous token: 'Balances'

/home/gour/financije/count.ledger:7:       Lexer error; erroneous token: ':eur'

/home/gour/financije/count.ledger:8:       Lexer error; erroneous token: ':hrk'

[...]

and the head (line #s added by me) of my count.ledger looks as follows:

#1 ; year 2015
#2
#3 ;include directives.ledger
#4
#5 2014/12/31 * Opening Balances
#6    Assets:ENC                                 59.48 HRK
#7    Assets:Gotovina:eur                      1088.60 EUR
#8    Assets:Gotovina:hrk                     1,470.95 HRK
#9 ...

Beancount has a more structured syntax & grammar than Ledger.

Your narration must be declared in a string, thus you need to quote it.

Also, account name components must start with a capital letter.

Try this for a working file:

plugin "beancount.ops.auto_accounts"

2014/12/31 * "Opening Balances"

   Assets:ENC                                 59.48 HRK

   Assets:Gotovina:Eur                      1088.60 EUR

   Assets:Gotovina:Hrk                     1,470.95 HRK

(The transaction does not balance and so you get a [correct] error.)

> Converting the other way has to be done manually so far.
> It might be worthwhile searching the mailing-list for hints on what others
> have done.

OK.

> Beancount does not currently support Unicode.
> I want to do it someday, but it'll require a lexer rewrite, and it's a low
> priority item.

Too bad. It seems it's showstopper for me since I annotate my ledger by using
native (Croatian) characters...

I know.

[Gour]

Martin Blais <bl...@furius.ca> writes:

> Beancount has a more structured syntax & grammar than Ledger.
> Your narration must be declared in a string, thus you need to quote it.

Ahh, OK.

> Also, account name components must start with a capital letter.

Strictness is good.

> Try this for a working file:
>
>
> plugin "beancount.ops.auto_accounts"

I tried example.beancount from the repo and I must say it looks very nice,
iow. I really like it. Congrats Martin for your accomplishment!!

> 2014/12/31 * "Opening Balances"
> Assets:ENC 59.48 HRK
> Assets:Gotovina:Eur 1088.60 EUR
> Assets:Gotovina:Hrk 1,470.95 HRK
>
> (The transaction does not balance and so you get a [correct] error.)

Well, I my case/file it does since I posted only part of it. :-)

>> Too bad. It seems it's showstopper for me since I annotate my ledger by
>> using native (Croatian) characters...

> I know.

I'll do some more study of provided docs and monitor your announcements in
ledger-cli checking whether/when Unicode-aware lexer will get some higher-priority
injection. ;)

[Martin Michlmayr]

* Gour <go...@atmarama.net> [2015-05-22 14:26]:

> > Beancount does not currently support Unicode. I want to do it
> > someday, but it'll require a lexer rewrite, and it's a low
> > priority item.
>
> Too bad. It seems it's showstopper for me since I annotate my ledger
> by using native (Croatian) characters...

I'd like to add my +1 for increasing the priority of Unicode support.

While I maintain my ledger mostly in English, I have non-ASCII
characters because of payee names, etc.

--
Martin Michlmayr
http://www.cyrius.com/

[redst...@gmail.com]

On Friday, May 22, 2015 at 5:27:13 AM UTC-7, Gour wrote:

Martin Blais <bl...@furius.ca> writes:

> There is an easy way to convert from Beancount to Ledger: bean-report has a
> command to do that:
>
>   bean-report <filename> ledger

It does not work here, iow. I get:

gour@atmarama ~/financije> bean-report count.ledger ledger
/home/gour/financije/count.ledger:5:       syntax error, unexpected ERROR

/home/gour/financije/count.ledger:5:       Lexer error; erroneous token: 'Opening'

/home/gour/financije/count.ledger:5:       Lexer error; erroneous token: 'Balances'

/home/gour/financije/count.ledger:7:       Lexer error; erroneous token: ':eur'

/home/gour/financije/count.ledger:8:       Lexer error; erroneous token: ':hrk'

[...]

and the head (line #s added by me) of my count.ledger looks as follows:

It seems like you are running bean-report on a ledger file. You can use bean-report only on a beancount file, to convert it to ledger (Martin said this too, but it seems to have been lost in communication). You seem to be trying the opposite, from what I can tell.

[Martin Blais]

Just had an idea while cycling: What if I did a Python-level translation that converts non-ascii chars to some "close" ascii equivalent, or just using 'ignore' decoding that replaces them by '?', send that output to a text file and parse that transparently?

It wouldn't be super great, but it would at least work and you could use the tools with those somewhat mangled strings, and later on when the correct support with a lexer arrives everything would just start working.

Would that be useful? It would take me 30 minutes to code that in with a test and release it.

Thoughts welcome,

[Gour]

Martin Blais <bl...@furius.ca> writes:

> Just had an idea while cycling: What if I did a Python-level translation
> that converts non-ascii chars to some "close" ascii equivalent, or just
> using 'ignore' decoding that replaces them by '?', send that output to a
> text file and parse that transparently?

Well, it would be *some* improvement...

>
> It wouldn't be super great, but it would at least work and you could use
> the tools with those somewhat mangled strings, and later on when the
> correct support with a lexer arrives everything would just start working.

...but I must say that, after using computers for quite some time and going
through times[1] of having to decide whether to write source code using strange
chars (š,Š, ž, Ž) OR not having native chars in the text output produced by
matrix printer followed by fiddling with codepages...I simply expect to have
proper Unicode (UTF_8) support in 21st century.

Lacking in this area *might* be the reason why Beancount is not (more)
popular since after skimming and/or reading part of the docs it's *very well*
designed[2] system which attracts me a lot providing ability to tweak it to
one's own needs with some knowledge of Python language.


Sincerely,
Gour

Footnotes:
[1] here I refer to using Hercules monographic cards

[2] e.g. I like that its syntax is more strict than the one in Ledger, less
options, web UI etc.

--
The working senses are superior to dull matter; mind is higher
than the senses; intelligence is still higher than the mind;
and he [the soul] is even higher than the intelligence.

[Gour]

Gour <go...@atmarama.net> writes:

> [2] e.g. I like that its syntax is more strict than the one in Ledger, less
> options, web UI etc.

I also like ability to have my ledger/beanaccount file in org-mode which I
use a lot. :.-)


Sincerely,
Gour

--

[Martin Blais]

On Fri, May 22, 2015 at 4:42 PM, Martin Michlmayr <t...@cyrius.com> wrote:

Actually, if it's just for string contents (e.g. narration, payee), it already works. Unicode in strings already parses through by lexer because when tokenizing a string it pretty much only looks for " and lets anything else through, which is being interpreted as UTF-8.

It's only outside of those strings, e.g., account names, tags, links, etc. that I need to do something special for unicodeo.

 

[Gour]

Martin Blais <bl...@furius.ca> writes:

> Actually, if it's just for string contents (e.g. narration, payee), it
> already works. Unicode in strings already parses through by lexer because
> when tokenizing a string it pretty much only looks for " and lets anything
> else through, which is being interpreted as UTF-8.

Hmm...that's quite different story then.

> It's only outside of those strings, e.g., account names, tags, links, etc.
> that I need to do something special for unicodeo.

From the above list, I'd have to change/tweak account names to accomodate
non-Unicode support which does not sound so bad.

Otoh, have to investigate within the docs about 'tags' in Beancount as well as
links etc.


Sincerely,
Gour

[Gour]

On Saturday, May 23, 2015 at 2:11:33 AM UTC+2, redst...@gmail.com wrote:

It seems like you are running bean-report on a ledger file. You can use bean-report only on a beancount file, to convert it to ledger (Martin said this too, but it seems to have been lost in communication). You seem to be trying the opposite, from what I can tell.

You're right. :-)

Now I've noticed the following in Tutorial doc:

bean-report ​ example.beancount ledger ​ > example.lgr

So, it would be nice to have some decent ledger --> beancount importer since I believe I'm going to switch and have ~9K transactions (>150 accounts etc.) in my ledger file (both in split and single version).

Any further hint how to migrate from Ledger?


Sincerely,
Gour

[Martin Blais]

Someone just needs to use the Ledger Python bindings to write out a Beancount-compatible data structure. It should rename things where necessary (e.g. $ -> USD or other, remove spaces in account names, etc.), but one of the issues is how to decide whether a conversion is intended to be made at-cost or by price, because in Beancount the semantics are different. Creative thinking will be required to do this, I think that the converter would need to be provided hints to carry out the translation properly. Ideally this should be all automatic.

Unfortunately, the converter couldn't leverage the Beancount codebase at the moment unless Ledger Python bindings support Python 3. Given that it uses Boost and Boost supports Python 3, this should be doable with a minimal amount of fiddling. If Python 2 only, I would make it just spit out text in the more simple version. If an all automatic solution is possible and Python 3 is supported, we could even envision creating a Beancount loader from Ledger syntax, whereby the user could potentially maintain their source in Ledger syntax for a while.

Just some ideas.

Sincerely,
Gour

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/480a3c6e-d24c-48a2-8417-02bd43f9f6d7%40googlegroups.com.

[Gour]

Martin Blais <bl...@furius.ca> writes:

> Someone just needs to use the Ledger Python bindings to write out a
> Beancount-compatible data structure. It should rename things where
> necessary (e.g. $ -> USD or other, remove spaces in account names, etc.),

In my case, I'm using HRK/EUR and not symbols, but would have to add dashes
for spaces in some accounts, which is, anyway, a good practice.

> but one of the issues is how to decide whether a conversion is intended to
> be made at-cost or by price, because in Beancount the semantics are
> different. Creative thinking will be required to do this, I think that the
> converter would need to be provided hints to carry out the translation
> properly. Ideally this should be all automatic.

I believe that I'll proceed 'manually' by using editor's (Emacs) features to
do the needful.

Otoh, Beancount seems to allow more logical organization of the COA, so
automatic conversion would have to be pretty smart. :-)

Otoh, it's one-time-only task - if I stay with Beancount, there is no need
for further work and if something would happen with the project there is,
fortunately, ability to switcb back by using 'ledger' export.

Now, I wonder whether I should convert just my present 2015.ledger counting
that Beancount will be able to include (old) files in the future or I its
performance should not be the problem so one is safe to keep everything
within single file?

Here is the stats of the single file which I have - not quite up-to-date, but
pretty close:

Time period: 10-Oct-31 to 15-Dec-17 (1873 days)

Files these postings came from:
/home/gour/financije/financije.ledger.org

Unique payees: 2681
Unique accounts: 151

Number of postings: 9026 (4.8 per day)
Uncleared postings: 126

Days since last post: -206
Posts in last 7 days: 30
Posts in last 30 days: 52
Posts seen this month: 8



Sincerely,
Gour

--
But for one who takes pleasure in the self, whose human life
is one of self-realization, and who is satisfied in the self only,
fully satiated — for him there is no duty.

[Martin Blais]

On Mon, May 25, 2015 at 2:43 AM, Gour <go...@atmarama.net> wrote:

Martin Blais <bl...@furius.ca> writes:

> Someone just needs to use the Ledger Python bindings to write out a
> Beancount-compatible data structure. It should rename things where
> necessary (e.g. $ -> USD or other, remove spaces in account names, etc.),

In my case, I'm using HRK/EUR and not symbols, but would have to add dashes
for spaces in some accounts, which is, anyway, a good practice.

No investments? If no investments, that should be very easy.

> but one of the issues is how to decide whether a conversion is intended to
> be made at-cost or by price, because in Beancount the semantics are
> different. Creative thinking will be required to do this, I think that the
> converter would need to be provided hints to carry out the translation
> properly. Ideally this should be all automatic.

I believe that I'll proceed 'manually' by using editor's (Emacs) features to
do the needful.

Personally I would write a Python script to automate the conversion and kick the tires on this, even if you might have to make small Ledger-compatible changes to your input source in order to make that possible. I think that gives you the flexibility to try it out, see if you like it, without having to commit to a full conversion.

 

Otoh, Beancount seems to allow more logical organization of the COA, so
automatic conversion would have to be pretty smart. :-)

I'm not sure what you mean by that, the only difference is that it requires your accounts to be in one of the five categories, and does not support "virtual" postings.

Otoh, it's one-time-only task - if I stay with Beancount, there is no need
for further work and if something would happen with the project there is,
fortunately, ability to switcb back by using 'ledger' export.

Note that the export-to-ledger command is pretty green; AFAIK I'm the only user of it (Bueller?). I haven't gotten any bug reports about it, so I'm guessing others aren't using it. You may run into minor issues. If you encounter any problems let me know.

Now, I wonder whether I should convert just my present 2015.ledger counting
that Beancount will be able to include (old) files in the future or I its
performance should not be the problem so one is safe to keep everything
within single file?

It can include old files, but if I were you I would start with just the 2015 file and try to generate the reports you're used to with Ledger before doing too much. In other words, I encourage you to test it out with a small file at first, especially if you're going to do the conversion manually. You may find some of the reports you're getting from Ledger aren't there, or run into some other problem. I would be conservative if I were you, since you already have a working Ledger workflow and corresponding expectations.

Here is the stats of the single file which I have - not quite up-to-date, but
pretty close:

Time period: 10-Oct-31 to 15-Dec-17 (1873 days)

  Files these postings came from:
    /home/gour/financije/financije.ledger.org

  Unique payees:            2681
  Unique accounts:           151

  Number of postings:       9026 (4.8 per day)
  Uncleared postings:        126

  Days since last post:     -206
  Posts in last 7 days:       30
  Posts in last 30 days:      52
  Posts seen this month:       8

That's small, you will not be concerned with performance.

(Note that I have yet to do any serious performance optimization, and so should it be needed I could cut down runtime by probably 50%, there are a few easy targets for this.)

Sincerely,
Gour

--
But for one who takes pleasure in the self, whose human life
is one of self-realization, and who is satisfied in the self only,
fully satiated — for him there is no duty.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/87vbfhdunu.fsf%40atmarama.ddns.net.

[Gour]

> No investments? If no investments, that should be very easy.

Don't forget I'm from Croatia and I do not recommend to invest here. :-)

I only have tiny price-history database consisting of several entries denotes
rates for currency exchange transactions.

> I think that gives you the flexibility to try it out, see if you
> like it, without having to commit to a full conversion.

Let me say that whatever I see in Beancount I simply like it. ;)

> It can include old files, but if I were you I would start with just the
> 2015 file and try to generate the reports you're used to with Ledger before
> doing too much. In other words, I encourage you to test it out with a small
> file at first, especially if you're going to do the conversion manually.

That's helpful hint in the sense that there will be less work to start using
Beancount regularly and then later include 'old' files.

> You may find some of the reports you're getting from Ledger aren't there,
> or run into some other problem. I would be conservative if I were you,
> since you already have a working Ledger workflow and corresponding
> expectations.

I am not using many Ledger reports considering I've recently converted to
Ledger from Gnucash...hledger's web view would be nice, but now I'm running
openSUSE which does not package it and very soon I'm going to Free/PC-BSD
where there is no package either and it looks that even Michael is fighting
with Haskell's tools, so Beancount seems 'just right'. :-)

> That's small, you will not be concerned with performance.

Nice to hear.

> (Note that I have yet to do any serious performance optimization, and so
> should it be needed I could cut down runtime by probably 50%, there are a
> few easy targets for this.)

Even better. ;)


Sincerely,
Gour

--
You have a right to perform your prescribed duty, but you
are not entitled to the fruits of action. Never consider
yourself the cause of the results of your activities,
and never be attached to not doing your duty.