-X option breaks things

[Yuri Khan]

Hello,

I deal with multiple currencies and occasionally I want to do external
processing on my journal as if everything was kept in my native
currency. More specifically, I try to use the Python binding like
this:

#!/usr/bin/python

from __future__ import print_function
import ledger

journal = ledger.read_journal('test.ledger')
for post in journal.query('-X RUB'):
print(post.date, post.account, post.amount)

However, it breaks on this minimal example (test1.ledger attached):

2019-02-09 payee 1
Expenses:Foo 10 USD
Assets:Bank card -69.54 RUB

2019-02-10 payee 2
Expenses:Bar 10 USD
Assets:Bank card -70 RUB

Specifically, it prints the two postings of the first transaction,
then errors out with:

RuntimeError: Assertion failed in
"/build/ledger-h93drX/ledger-3.1.2~pre1+g3a00e1c+dfsg1/src/item.h",
line 181:virtual ledger::date_t ledger::item_t::date() const: _date

I tracked it down to there being a dummy transaction between days when
a currency price changes:

```
$ ledger print -f test1.ledger -X RUB
2019-02-09 payee 1
Expenses:Foo 10 USD
Assets:Bank card -69.54 RUB

2019-02-10 Commodities revalued

2019-02-10 payee 2
Expenses:Bar 10 USD
Assets:Bank card -70.00 RUB
```

That “Commodities revalued” transaction causes the .query iterator to
yield a ledger.JournalItem object instead of a ledger.Posting, and
asking that JournalItem for a date triggers an assertion.

I tried then to ignore any objects other than Posting:

for post in journal.query('-X RUB'):
if type(post) is ledger.Posting:
print(post.date, post.account, post.amount)

It works on the simple two-transaction example, but, as the number of
transactions (or revaluations?) grows, at some point it just
segfaults. Nine transactions trigger the segfault for me (test.ledger
attached).

I also tried to work around the whole Ledger Python binding by
exporting XML. It also breaks on revaluations:

$ ledger xml -X RUB -f test.ledger
While handling posting from "/home/yuri/src/ledger/test.ledger", line 6:
> Expenses:Bar 10 USD
Error: Assertion failed in
"/build/ledger-h93drX/ledger-3.1.2~pre1+g3a00e1c+dfsg1/src/ptree.cc",
line 97:virtual void
ledger::format_ptree::operator()(ledger::post_t&):
post.xdata().has_flags(POST_EXT_VISITED)

What else can I try in order to get a stream of postings normalized to
a single currency? (`ledger reg -X RUB` works but I would very much
dislike to scrape its output.)

[John Wiegley]

>>>>> "YK" == Yuri Khan <yuri....@gmail.com> writes:

YK> What else can I try in order to get a stream of postings normalized to a
YK> single currency? (`ledger reg -X RUB` works but I would very much dislike
YK> to scrape its output.)

Does --no-revalued help with the query?

John

[Yuri Khan]

I was thinking I tried that, but it turns out I didn’t. Yes,
--no-revalued affects all of: ledger print, ledger xml, and
legder.Journal.query. Thank you.

Now it appears what I asked for is not what I need :) So I will
describe the bigger picture.

I have a Python script that takes a Ledger journal and a query, and
produces a stacked graph of each account’s balance over time.
Currently, it supports only a single currency, ignoring all the
others.

https://github.com/yurikhan/ledger-d3/

I am currently extending it so that it can track investments. I was
thinking I could add -X RUB --no-revalued to the query, but that
freezes each investment’s price to the value it had at the time of
purchase. I want to track currency prices throughout the time range.

To that end, I calculate partial sums in each currency separately,
which gives me a mapping (date, account, currency) → balance. Then, in
order to plot the graph, I need to know the price of each currency at
each given date.

I have found ledger.Commodity.find_price which accepts a base
Commodity and two time points, but I seem to be unable to use it from
Python:

#!/usr/bin/python

from datetime import datetime
import ledger

journal = ledger.read_journal('test1.ledger')
usd = ledger.commodities.find('USD')
rub = ledger.commodities.find('RUB')
print(usd.find_price(rub, datetime(2019,2,1), datetime(1970,1,1))

Traceback (most recent call last):
File "./test_price.py", line 9, in <module>
print(usd.find_price(rub, datetime(2019,2,1), datetime(1970,1,1)))
TypeError: No to_python (by-value) converter found for C++ type:
boost::optional<ledger::price_point_t>

There are more things in the Python API that use unexposed types:

* AutomatedTransaction.extend_xact accepts a parse_context_t as its
second argument
* PeriodicTransaction.period has type date_interval_t


PS: I was able to solve my problem using Amount.value(Commodity,
date). Still, having to backtrack from that find_price dead end took
me some time.

[John Wiegley]

>>>>> "YK" == Yuri Khan <yuri....@gmail.com> writes:

YK> PS: I was able to solve my problem using Amount.value(Commodity, date).
YK> Still, having to backtrack from that find_price dead end took me some
YK> time.

Yes, the Python API is still a bit experimental, so it doesn't surprise me
there are types still not being marshaled. They aren't terribly hard to add,
at least.

John