Charts software for ledger?

[Manuel Amador (Rudd-O)]

Hi! I'm looking for a program that will help me chart my net worth and
other such things such as expenses.

The few programs I have found have deplorable installation instructions
("run this program, it will download some mystery meat from the
Internets, then put some files here, then create a configuration file")
and have questionable security practices ("then run this other program,
which starts an HTTP server on your local machine").

Nothing of the sort is necessary if a good desktop application is
written using libraries that are readily available in most Linux
distributions. To wit, my ledgerhelpers helpers are desktop apps, they
require no configuration at all, and they work with your existing setup
(modulo a few format limitations).

Why isn't there anything like that? How hard can it be to load data
from ledger using its Python bindings, then plot it using something like
pygtkchart or kqtquickcharts?

Anyway, just my thoughts.

--
Rudd-O
http://rudd-o.com/

[Martin Blais]

On Wed, Apr 20, 2016 at 3:43 PM, Manuel Amador (Rudd-O) <rud...@rudd-o.com> wrote:

Hi!  I'm looking for a program that will help me chart my net worth and
other such things such as expenses.

The few programs I have found have deplorable installation instructions
("run this program, it will download some mystery meat from the
Internets, then put some files here, then create a configuration file")
and have questionable security practices ("then run this other program,
which starts an HTTP server on your local machine").

Nothing of the sort is necessary if a good desktop application is
written using libraries that are readily available in most Linux
distributions.  To wit, my ledgerhelpers helpers are desktop apps, they
require no configuration at all, and they work with your existing setup
(modulo a few format limitations).

Why isn't there anything like that? 

You mean like this?

https://bitbucket.org/blais/beancount/src/8ac805ab793f96594871a1594e72f4039bbc2ee3/experiments/compensation/net-worth-over-time.py?at=default&fileviewer=file-view-default

[Craig Earls]

Since i have never gotten python to work without segfaults i cant answer

--

---
You received this message because you are subscribed to the Google Groups "Ledger" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ledger-cli+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Craig, Corona De Tucson, AZ
enderw88.wordpress.com

[Manuel Amador (Rudd-O)]

Pretty much. Does this mean I must switch to Beancount?


--
Rudd-O
http://rudd-o.com/

[Manuel Amador (Rudd-O)]

On 04/20/2016 09:00 PM, Craig Earls wrote:
> Since i have never gotten python to work without segfaults i cant answer

My trick is to load a new Journal object after a Query, otherwise it
segfaults.

--
Rudd-O
http://rudd-o.com/

[Manuel Amador (Rudd-O)]

On 04/20/2016 09:00 PM, Craig Earls wrote:

> Since i have never gotten python to work without segfaults i cant answer

After writing a small app that iterates through Ledger transactions and
plots them, it has become exceedingly clear that you were right. Simply
going over a query() result causes it to crash or to come up with absurd
errors like "Year must be between 1400...10000". Furthermore, for some
reason now my GUI apps that used to work throw out balance errors.

Really shoddy quality. Can't wait to switch to Beancount.

--
Rudd-O
http://rudd-o.com/

[Ben Finney]

"Manuel Amador (Rudd-O)"

<rud...@rudd-o.com> writes:

> On 04/20/2016 09:00 PM, Craig Earls wrote:
> > Since i have never gotten python to work without segfaults i cant
> > answer

We are talking about the Python API supplied as part of the Ledger
distribution?

> After writing a small app that iterates through Ledger transactions and
> plots them, it has become exceedingly clear that you were right. Simply
> going over a query() result causes it to crash or to come up with absurd
> errors like "Year must be between 1400...10000".

Can you direct us to the bug reports detailing these problems?

--
\ “A thing moderately good is not so good as it ought to be. |
`\ Moderation in temper is always a virtue; but moderation in |
_o__) principle is always a vice.” —Thomas Paine |
Ben Finney

[Craig Earls]

I have been using ledger happily for five years now. I just havent put much effort into. 

--

---
You received this message because you are subscribed to the Google Groups "Ledger" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ledger-cli+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Manuel Amador (Rudd-O)]

On 04/20/2016 11:23 PM, Ben Finney wrote:
> "Manuel Amador (Rudd-O)"
> <rud...@rudd-o.com> writes:
>
>> On 04/20/2016 09:00 PM, Craig Earls wrote:
>>> Since i have never gotten python to work without segfaults i cant
>>> answer
> We are talking about the Python API supplied as part of the Ledger
> distribution?

Yes.

>
>> After writing a small app that iterates through Ledger transactions and
>> plots them, it has become exceedingly clear that you were right. Simply
>> going over a query() result causes it to crash or to come up with absurd
>> errors like "Year must be between 1400...10000".
> Can you direct us to the bug reports detailing these problems?
>

Old report is available as one of my pull requests in the ledger/ledger
Github repo. Which, by the way, remains unmerged.

Sigh.

--
Rudd-O
http://rudd-o.com/

[John Wiegley]

>>>>> Manuel Amador (Rudd-O) <rud...@rudd-o.com> writes:

> Old report is available as one of my pull requests in the ledger/ledger
> Github repo. Which, by the way, remains unmerged.

Sorry about that; merged now.

--
John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2

[Metin Akat]

This is a bug report that shows how/when/why it crashes:  http://bugs.ledger-cli.org/show_bug.cgi?id=1163

The same thing happens (for me) when I access it via Python

It only happens with larger files (mine has ~20k transactions).

[Lifepillar]

I don't know whether this might fit what you're looking after, but the
approach I have been using for a while (and which I am satisfied with) is
to let Ledger output the data in tabular format, then feed the output to
some plotting program. With some scripting, this is easily automated.
I use R, but you may as well use whatever app able to read CSV (including
{Libre|Open}Office). As an example, my Ledger bundle for TextMate is able
to plot charts if R is installed
(https://github.com/lifepillar/Ledger.tmbundle).
TextMate is OS X only, though.

To export the data in a way that I may draw nearly any imaginable plot, I
use three formats. One for register/periodic reports, with these fields:

date; year; month; month_num; wday; wday_num; week; mday; amount;
total; payee; account

which is generated with this option (one line):

--format
'%(format_date(date,\"%Y-%m-%d;%Y;%b;%m;%a;%u;%W;%d\"));%(quantity(scrub(display_amount)));%(quantity(scrub(display_total)));%(payee);%(display_account)\n'


One for balance reports, with these fields:

balance; uncleared; account; partial_account

--format '%(quantity(scrub(get_at(display_total,
0))));%(quantity(scrub(get_at(display_total,
1))));%(account);%(partial_account)\n%/'

Finally, one for budget reports, with these fields:

actual; budgeted; remaining; used; account; partial_account

--format '%(quantity(scrub(get_at(display_total,
0))));%(get_at(display_total, 1) ?
quantity(-scrub(get_at(display_total, 1))) :
0.0);%(get_at(display_total, 1) ? (get_at(display_total, 0) ?
quantity(-scrub(get_at(display_total, 1) + get_at(display_total, 0))) :
quantity(-scrub(get_at(display_total, 1)))) :
quantity(-scrub(get_at(display_total, 0))));%(get_at(display_total, 1)
? quantity(100% * (get_at(display_total, 0) ?
scrub(get_at(display_total, 0)) : 0.0) / -scrub(get_at(display_total,
1))) : "na");%(account);%(partial_account)\n%/'

They are horribly complicated, but they get the job done beautifully. I
do not type them all the time, of course: I use a Vim command that
prints a report in any of those formats.

Enjoy,
Life

[Jeffrey McBeth]

I do something similar with orgmode being the intermediary between ledger and pandas

[Martin Blais]

That's not a constructive way to help.

Pissing off Ledger developers doesn't help anybody.

The software's free; there's probably just a bug in its Python bindings, it happens, situation normal.

The best attitude is sending a patch to fix the bug.

[Manuel Amador (Rudd-O)]

On 04/22/2016 02:28 PM, Martin Blais wrote:
> On Wed, Apr 20, 2016 at 6:55 PM, Manuel Amador (Rudd-O)
> <rud...@rudd-o.com <mailto:rud...@rudd-o.com>> wrote:
>
> On 04/20/2016 09:00 PM, Craig Earls wrote:
> > Since i have never gotten python to work without segfaults i cant answer
>
> After writing a small app that iterates through Ledger
> transactions and
> plots them, it has become exceedingly clear that you were right.
> Simply
> going over a query() result causes it to crash or to come up with
> absurd
> errors like "Year must be between 1400...10000". Furthermore, for
> some
> reason now my GUI apps that used to work throw out balance errors.
>
> Really shoddy quality. Can't wait to switch to Beancount.
>
>
> That's not a constructive way to help.

I wasn't trying to help. I was venting.

Not that I am unhelpful. On the contrary, I help the project by sending
patches of whatever stuff I /can/ code, and I also try to help the
project by writing utilities and forming an ecosystem around the project
(this is the primary way in which I try to contribute, because I /do/
like the ideas and principles behind the project). But, as it so
happens, C++ is not one of those things I can do, and the (obviously
long-standing) bug in the Python bindings is preventing me from growing
that ecosystem I spoke of.

And, as you can see from other replies, I'm not the only one who feels
badly about the Python bindings crashing and burning. Who knows how
many programs to support Ledger did /not/ get written because of this
quality issue? We won't know.

But, I /do/ know I am going to cease development of my own
ecosystem-enhancing utilities until the issue is resolved (which I hope
someone else does because, as I have explained, I do not know how).
After all, what's the point of writing a program which is going to
segfault on the user (including me)?

--
Rudd-O
http://rudd-o.com/

[Manuel Amador (Rudd-O)]

On 04/21/2016 05:20 AM, Metin Akat wrote:
> This is a bug report that shows how/when/why it crashes:
> http://bugs.ledger-cli.org/show_bug.cgi?id=1163
> The same thing happens (for me) when I access it via Python
> It only happens with larger files (mine has ~20k transactions).
>

YES! This is exactly my crash.

My file has grown to about ~10K lines. If I am to do more than one
query operation using the bindings, Ledger crashes. Sometimes a single
query operation is enough (I can whip up some demo code, but I need to
anonymize my file).

Furthermore, I have a separate issue. With my file, the first Python
bindings query operation returns fine, but rereading the file and
issuing the exact same query barfs, saying some records do not balance
at all (by minuscule amounts, every time). Really problematic. I
believe it might be because later transactions in the file increase the
precision of the U.S. dollar, so the second time around that the file is
queried / loaded, the original transactions which balanced fine (with
the precision as-it-was-known-in-their-position-in-the-file) no longer
balance correctly. In my opinion, this should be considered broken
behavior.

--
Rudd-O
http://rudd-o.com/

[Manuel Amador (Rudd-O)]

On 04/21/2016 04:28 AM, John Wiegley wrote:
>>>>>> Manuel Amador (Rudd-O) <rud...@rudd-o.com> writes:
>> Old report is available as one of my pull requests in the ledger/ledger
>> Github repo. Which, by the way, remains unmerged.
> Sorry about that; merged now.

Thanks.

--
Rudd-O
http://rudd-o.com/

[Manuel Amador (Rudd-O)]

Thanks. I don't want commands running through CSV files. I want an
interactive tool that has things I can poke at, which will live-update
the graphs, show me legends, and allow me to interact with the graphs.

Since I could not find any, I HAD started writing one just two weeks ago
(my program uses pyplot provisionally).

I have ceased development of that program, and pretty much everything
until ledger's Python bindings stop making the program segfault.

So I guess I will have to wait for a very long time, and possibly
migrate to another accounting tool that I can commit to using.

--
Rudd-O
http://rudd-o.com/

[John Wiegley]

>>>>> Manuel Amador (Rudd-O) <rud...@rudd-o.com> writes:

> And, as you can see from other replies, I'm not the only one who feels badly
> about the Python bindings crashing and burning. Who knows how many programs
> to support Ledger did /not/ get written because of this quality issue? We
> won't know.

The Python bindings are indeed a weak point within the implementation of
C++Ledger. If anyone wants to take over maintenance of that part of the code,
they would be most welcome. Until then, I don't think that state of affairs
will change anytime soon.

You may prefer Beancount, which is implemented in Python.

[Martin Blais]

BTW have you seen Fava?

http://fava.pythonanywhere.com/

[Manuel Amador (Rudd-O)]

On 04/26/2016 02:42 PM, Martin Blais wrote:
> Thanks. I don't want commands running through CSV files. I want an
>
> interactive tool that has things I can poke at, which will live-update
> the graphs, show me legends, and allow me to interact with the graphs.
>
>
> BTW have you seen Fava?
> http://fava.pythonanywhere.com/

Oh.

That looks AMAZING.

Thank you! Even better reason to move to Beancount.

--
Rudd-O
http://rudd-o.com/

[Manuel Amador (Rudd-O)]

On 04/26/2016 02:39 PM, John Wiegley wrote:
>
> The Python bindings are indeed a weak point within the implementation of
> C++Ledger. If anyone wants to take over maintenance of that part of the code,
> they would be most welcome. Until then, I don't think that state of affairs
> will change anytime soon.
>
> You may prefer Beancount, which is implemented in Python.

For purely selfish reasons (less work for me), I would prefer the Python
bindings to be better with Ledger. It does look like Beancount is
winning here... and in many other senses.

--
Rudd-O
http://rudd-o.com/

[Ben Finney]

"Manuel Amador (Rudd-O)"
<rud...@rudd-o.com> writes:

> I would prefer the Python bindings to be better with Ledger. It does
> look like Beancount is winning here... and in many other senses.

What are the current problems with the Python API? Are they tracked
systematically somewhere, such that motivated people can work to address
them and know they're addressing real reported problems?

--
\ “In general my children refuse to eat anything that hasn't |
`\ danced on television.” —Erma Bombeck |
_o__) |
Ben Finney

[John Wiegley]

>>>>> Ben Finney <ben+l...@benfinney.id.au> writes:

> What are the current problems with the Python API? Are they tracked
> systematically somewhere, such that motivated people can work to address
> them and know they're addressing real reported problems?

I think we don't presently know the full extent of the problems, so a better
test suite would be a good place to start.

[Metin Akat]

Hi, and sorry for hijacking this thread.

It's not *only* a Python problem as I have demonstrated here: http://bugs.ledger-cli.org/show_bug.cgi?id=1163

The ledger C++ core crashes, not the python bindings.

For me it happens while interacting with Ledger's interactive prompt, AFAIK there is no python here at all.

I am happy to help with more testing, but regrettably, I too don't speak C++

Metin

[Ben Finney]

"John Wiegley" <jo...@newartisans.com> writes:

> >>>>> Ben Finney <ben+l...@benfinney.id.au> writes:
>
> > What are the current problems with the Python API? Are they tracked
> > systematically somewhere, such that motivated people can work to address
> > them and know they're addressing real reported problems?
>
> I think we don't presently know the full extent of the problems, so a
> better test suite would be a good place to start.

Okay. To be clear, what Python API are we talking about? I assume this
means the ‘ledger’ Python module installed, whose source is the C++
files under ‘src/’. Yes?

What constitutes the current test suite for that code, and how is it
automated?

--
\ “We can't depend for the long run on distinguishing one |
`\ bitstream from another in order to figure out which rules |
_o__) apply.” —Eben Moglen, _Anarchism Triumphant_, 1999 |
Ben Finney

[Yuval Hager]

On 27/04/16 03:08 AM, Metin Akat wrote:
> Hi, and sorry for hijacking this thread.
>
> It's not *only* a Python problem as I have demonstrated here:
> http://bugs.ledger-cli.org/show_bug.cgi?id=1163
> The ledger C++ core crashes, not the python bindings.
> For me it happens while interacting with Ledger's interactive prompt, AFAIK
> there is no python here at all.
>

BTW, I have tried multiple times to reproduce the issue mentioned in the bug,
but got no crashes.

[Metin Akat]

I suspect it only happens with large journal files and large reports ( ones which return lots of lines of data). Will try and provide a journal file which causes it to happen.

[John Wiegley]

>>>>> Ben Finney <ben+l...@benfinney.id.au> writes:

> Okay. To be clear, what Python API are we talking about? I assume this means
> the ‘ledger’ Python module installed, whose source is the C++ files under
> ‘src/’. Yes?

It's in the py*.cc files under src/.

> What constitutes the current test suite for that code, and how is it
> automated?

The current tests are auto-generated from the C++ tests, so that the testing
should be identical for the covered modules.

[Ben Finney]

"John Wiegley" <jo...@newartisans.com> writes:

> >>>>> Ben Finney <ben+l...@benfinney.id.au> writes:
>

> > What constitutes the current test suite for that code, and how is it
> > automated?
>
> The current tests are auto-generated from the C++ tests, so that the
> testing should be identical for the covered modules.

Can you point me to instructions on how I would run the test suite
before and after a change? I'm quite new to this code base.

--
\ “Any intelligent fool can make things bigger and more complex… |
`\ It takes a touch of genius – and a lot of courage – to move in |
_o__) the opposite direction.” —Albert Einstein |
Ben Finney

[John Wiegley]

>>>>> Ben Finney <ben+l...@benfinney.id.au> writes:

> Can you point me to instructions on how I would run the test suite before
> and after a change? I'm quite new to this code base.

I usually do this as part of the acprep build. Perhaps others here are
familiar with just running the tests alone. It's been a while since I've done
that.