Inconsistent results.

[Charles Merriam]

I've been noticing ledger giving different results at random.  These are two identical commands run one after another with different results.

This is on ledger Ledger 3.0.1-20140327 from when I last did any development.   

Any hints?

[big2 *master] ~/p/fin (18682)$ ledger -f 2014.led -ww --sort date -E register Assets\:Star

13-Dec-31 Opening Balances                   Assets:StarOne Savings                            $12,069.70           $12,069.70

14-Jan-01 Balance $12069.70                  Assets:StarOne Savings                                     0           $12,069.70

14-Jan-09 Visa Rebate ; some sort of odd p.. Assets:StarOne Savings                                 $4.26           $12,073.96

14-Jan-16 Transfer from Checking             Assets:StarOne Savings                               $337.58           $12,411.54

14-Jan-31 Interest on Savings                Assets:StarOne Savings                                 $6.76           $12,418.30

14-Jan-31 Balance $12418.30                  Assets:StarOne Savings                                     0           $12,418.30

14-Feb-14 Overpayment from StarVisa          Assets:StarOne Savings                               $472.51           $12,890.81

14-Feb-28 Interest on Savings                Assets:StarOne Savings                                 $6.32           $12,897.13

14-Feb-28 Balance $12897.13                  Assets:StarOne Savings                                     0           $12,897.13

14-Mar-14 Overpayment from StarVisa          Assets:StarOne Savings                               $457.51           $13,354.64

14-Mar-31 Interest on Savings                Assets:StarOne Savings                                 $7.27           $13,361.91

14-Mar-31 Balance $13361.91                  Assets:StarOne Savings                                     0           $13,361.91

14-Apr-16 Transfer to StarOne                Assets:StarOne Savings                               $448.79           $13,810.70

14-Apr-30 Interest on Savings                Assets:StarOne Savings                                 $7.26           $13,817.96

14-Apr-30 Balance $13817.96                  Assets:StarOne Savings                                     0           $13,817.96

14-May-07 Check towards TechCu               Assets:StarOne Savings                           $-12,000.00            $1,817.96

14-May-16 Transfer to StarOne                Assets:StarOne Savings                                $60.02            $1,877.98

14-May-31 Interest on Savings                Assets:StarOne Savings                                $-2.59            $1,875.39

14-May-31 Balance $2118.24                   Assets:StarOne Savings                                     0            $1,875.39

[big2 *master] ~/p/fin (18683)$ ledger -f 2014.led -ww --sort date -E register Assets\:Star

13-Dec-31 Opening Balances                   Assets:StarOne Savings                            $12,069.70           $12,069.70

14-Jan-01 Balance $12069.70                  Assets:StarOne Savings                                     0           $12,069.70

14-Jan-09 Visa Rebate ; some sort of odd p.. Assets:StarOne Savings                                 $4.26           $12,073.96

14-Jan-16 Transfer from Checking             Assets:StarOne Savings                               $337.58           $12,411.54

14-Jan-31 Interest on Savings                Assets:StarOne Savings                                 $6.76           $12,418.30

14-Jan-31 Balance $12418.30                  Assets:StarOne Savings                                     0           $12,418.30

14-Feb-14 Overpayment from StarVisa          Assets:StarOne Savings                               $472.51           $12,890.81

14-Feb-28 Interest on Savings                Assets:StarOne Savings                                 $6.32           $12,897.13

14-Feb-28 Balance $12897.13                  Assets:StarOne Savings                                     0           $12,897.13

14-Mar-14 Overpayment from StarVisa          Assets:StarOne Savings                               $457.51           $13,354.64

14-Mar-31 Interest on Savings                Assets:StarOne Savings                                 $7.27           $13,361.91

14-Mar-31 Balance $13361.91                  Assets:StarOne Savings                                     0           $13,361.91

14-Apr-16 Transfer to StarOne                Assets:StarOne Savings                               $448.79           $13,810.70

14-Apr-30 Interest on Savings                Assets:StarOne Savings                                 $7.26           $13,817.96

14-Apr-30 Balance $13817.96                  Assets:StarOne Savings                                     0           $13,817.96

14-May-07 CoinStar Jar of Coins              Assets:StarOne Savings                                $27.69           $13,845.65

                                             Assets:StarOne Savings                                 $7.96           $13,853.61

                                             Assets:StarOne Savings                               $156.89           $14,010.50

                                             Assets:StarOne Savings                                $45.13           $14,055.63

14-May-07 Check towards TechCu               Assets:StarOne Savings                           $-12,000.00            $2,055.63

14-May-16 Transfer to StarOne                Assets:StarOne Savings                                $60.02            $2,115.65

14-May-31 Interest on Savings                Assets:StarOne Savings                                $-2.59            $2,113.06

14-May-31 Balance $2118.24                   Assets:StarOne Savings                                     0            $2,113.06

[big2 *master] ~/p/fin (18683)$ ledger -V

Ledger 3.0.1-20140327, the command-line accounting tool

This offending transaction was:

2014/05/07   CoinStar Jar of Coins

    cash     -$27.69

    starone   $27.69

    cash     -$7.96

    starone   $7.96

    cash     -$156.89

    starone   $156.89

    cash      -$45.13

    starone    $45.13

but I seem to see it on many transactions.

[Craig Earls]

I am confused by your question. Are you saying there were no changes
to the data file in-between running those commands?

> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "Ledger" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ledger-cli+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Craig, Corona De Tucson, AZ
enderw88.wordpress.com

[Craig Earls]

I also don't recognize -ww as a valid option.

[Charles Merriam]

Yes.

[Charles Merriam]

Having fought with this more today, without digging into the source:

1.  Ledger does give different results for the same input file.   It seems to switch randomly. 

2.  Ledger fails by randomly failing to handle aliases from the included file.  In include a file 'accounts.led' which has all my aliases.

3.  The easiest test case for me is to use '--strict'.   I will fail about 35% of the time with each alias being an unknown account.

4.  The first lines of my 2014 register:

;* WELCOME to 2014

include accounts.led

;*  Opening balances

Is this a known problem?  Is there a work-around?

-- Charles

[Craig Earls]

To my knowledge we have not see this before. Can you put together a set of file that exhibit this failure that you are willing to share?

--

---
You received this message because you are subscribed to the Google Groups "Ledger" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ledger-cli+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Charles Merriam]

Hi Craig,

Thank you. At this time, it would not be worth digging into it
further without actually digging into the code to fix it.
Unfortunately, my previous experience suggests it would not be
worthwhile fixing it unless a patch would be accepted.

Thank you for letting me know it is not a known bug.

Charles

> You received this message because you are subscribed to a topic in the
> Google Groups "Ledger" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/ledger-cli/Ol0W6BpabPw/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

[Craig Earls]

This would be very high priority for work. But we can't fix it if we can't duplicate it.  We are not asking you to dig into the source.  If you are willing to let us use your current data file that would help greatly. 

[Martin Michlmayr]

Hi Charles,

Are you by any chance on MacOS and using ledger from homebrew?

Someone mentioned a problem on IRC a few weeks ago that sounds similar
to yours.

Here are parts of the conversion:

20:41 < schmooser> one half of times I got 4 rows and another half of times I got 5 rows
20:45 <@tbm> schmooser: I always get the same result. What output do you get?
20:47 < schmooser> tbm: here is the screenshot http://img.ly/yTlu
20:47 < schmooser> tbm: ledger 3.0.2, compiled by homebrew on mavericks
20:53 < schmooser> tbm: huh, I have another copy of ledger, which I compiled from sources, not homebrew. It works normally.
20:54 <@tbm> schmooser: maybe homebrew uses clang to compile; someone reported issued with ledger comipled with clang on the list a few days ago



* Charles Merriam <charles...@gmail.com> [2014-06-23 17:34]:

Martin Michlmayr
http://www.cyrius.com/

[Martin Blais]

He could isolate the problem by copying the file and chopping it down, removing parts bit by bit until the problem stops manifesting, and only share and anonymify that minimal snippet.

[Craig Earls]

I use ledger on MacOS from home-brew and never see this. But I never
use the default register format and I have very few transactions with
more than two balancing postings.

[Harshad RJ]

On Tue, Jun 24, 2014 at 7:34 AM, Martin Blais <bl...@furius.ca> wrote:

anonymify that minimal snippet.

​Martin, that gives me an idea. Since beancount already has the ability to export transactions in ledger format, you could add a switch to obfuscate the account names, etc while exporting.​ It will make it easier to report bugs.

--

Harshad RJ

[Craig Earls]

Ledger has an anonymize function, but it does not preserve account
structure, it just puts random strings into every account space. This
has some drawbacks with bug reporting.

[Martin Blais]

That is an absolutely fantastic idea and I will implement it. I'm not convinced that just replacing account names will be enough to share one's file, but why not, it's easy. I'll try to think of a way to randomize some numbers safely.

Another idea I have been thinking about recently is "fuzzing": automatically generating a massive input file with all kinds of possible combinations to see where the system falls on its face or bring up bugs I may not have tested or. This is pretty common practice in protocol testing.

BTW, there is no conversion script yet, but it's one of the first things I want to do after completing the docs. (It's probably just a few hours of work in total to be honest and it'll be fun to play with other systems using my big corpus of data. The only complications lie in resolving difference in semantics - printing out compatible syntax would take 10 minutes, really).

Thanks,

[Charles Merriam]

Yes, I blew away my development version and use home-brew now.

[Martin Blais]

To reliably reproduce the problem, run the code in attachment.

You will see about 1/4th of the outputs are empty.

If you remove the alias, the problem "mostly" goes away;

if you do, sometimes you will get a rare empty file too (about 1-2/1000).

[Craig Earls]

Thanks I will start looking. Unstable sort or not, don't like this...

[Craig Earls]

I have localized it to the alias command as you suggested. Sort and
explicit have no effect on it. I am on Mavericks building from local
source. I will try different compilers.