What causes startup time of beancount?

[bw4...@googlemail.com]

I have build a script which basically sums up outstanding expenses for a specific customer account.

Each time I call this script, I feel it takes too long to execute. time says         0.18 real         0.16 user         0.01 sys

bean-query has similar performance, especially if I call it several times in a script.

Is this specific to my environment? Who can confirm a similar experience?

[Martin Blais]

The load time varies depending on the size of the file.

Mine takes about three seconds right now, as I maintain my entire history in a single file.

If you want to see the breakdown of which stages take how long, run

  bean-check -v <filename>

I have spent almost no time optimizing anything so far, but I'm confident I will eventually be able to cut the load time in half. I think a sensible goal would be to ensure 10 years worth of typical transactions parse in under 1 sec.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/4c19e464-3e87-41ec-941d-80ace9f9eb7a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[redst...@gmail.com]

Hi Martin,

So my beancount file takes a humongous time to load: about 2 minutes.

bean-check -v shows:

INFO    : Operation: 'beancount.parser.parser'                        Time: 1412 ms

INFO    : Operation: 'beancount.ops.balance'                          Time: 9500 ms

INFO    : Operation: 'beancount.loader (total)'                       Time: 110842 ms

My hacky solution this far has been to largely disable one of my source files (I have one file per account), which contains about 1.5k transactions out of a total of about 30k transactions. This cuts it down to about 18 seconds. The 1.5k transactions belong to an investment account and thus have stock purchases (and a few sells). If I remove all the balance assertions in it, the beancount.loader (total) goes to about 95 seconds.

I used a python profiler to further understand this: https://ibb.co/iCTjow

 61% of the time goes in 'add_amount()'. I'll take a deeper look later on, but thought I'd post here and see if you had any feedback.

Thanks!

[Martin Blais]

Can you try this patch?

https://bitbucket.org/blais/beancount/issues/191/beancountcoreinventory-should-be-a-dict

To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/3fd100db-40f4-4157-8a78-eb31f4248c05%40googlegroups.com.

[redst...@gmail.com]

Thanks for the pointer. Much nicer with the patch:

INFO    : Operation: 'beancount.parser.parser'                        Time:         1415 ms   

INFO    : Operation: 'beancount.ops.balance'                          Time:         1054 ms

INFO    : Operation: 'beancount.loader (total)'                       Time:  33385 ms

I'd like to eventually see if I can help get this down to under 10 sec, which makes it more usable. I'm running on a Core i7.

[Martin Blais]

Let's profile further, and if needed, to write the parts most called in C.

To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/7c3e801f-565e-4fe2-9d50-9054bb6b23f6%40googlegroups.com.

[redst...@gmail.com]

Sounds good. Here is the new profile after it came down from 110s to 33s: http://ibb.co/j62cQb

I'll take a closer look later.

[Martin Blais]

This copy is costly but might be avoidable, but it will perhaps be sensitive to remove it:

https://bitbucket.org/blais/beancount/src/6b61612e7f44580a9017c3c05d3efdff2981be81/beancount/parser/booking_full.py?at=default&fileviewer=file-view-default#booking_full.py-553

To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/7d9809a0-65b2-424d-a4b6-619c46343303%40googlegroups.com.