Proposal for cost accounting in transfers

[Eric Altendorf]

Hi all,

A few recent (and older threads) have discussed "transferring" cost information from account to account in a transfer transaction (reduction in one account with a balancing augmentation in another account).

I've studied the code and current behavior and it seems quite feasible to improve how this works.  I wrote up my findings and proposal here:

https://docs.google.com/document/d/1M0W9U5ZdwkiziLNKfiQVIHYSI7gz_gM9pMQhicNXbE0/edit?usp=sharing

I am sure my understanding of the system is limited and flawed, however, so I would greatly appreciate feedback, on my understanding of the problem and the current behavior, as well as on the proposed changes.

Thank you in advance,

eric

[Eric Altendorf]

Well, I actually just realized that I can't assume there is only one reduction posting, because the reduction may pull from multiple lots.  I need to investigate more.

[Eric Altendorf]

So having thought about this issue some more, I think I need some advice on a core architectural question.

We want to fill out cost information on an augmenting (receiving) leg of a transfer, such that it matches the cost information on the reducing legs of the transfer.  Should this be part of:

1) booking, because booking fills out cost information, and in particular, here we are filling out exactly the same cost information on the augmenting legs that booking is computing for the reducing legs,

or

2) imputation, because imputation is about inferring values to make the transaction balance, and to make a transaction balance, the augmenting cost info must match the reducing cost info

Conceptually, making it part of imputation feels more correct, but booking is already filling out some cost info (the date) and we'd have to undo that in imputation -- implementation wise it feels easier to implement this in booking.

[Martin Blais]

On Sat, Aug 19, 2023, 21:14 Eric Altendorf <erical...@gmail.com> wrote:

So having thought about this issue some more, I think I need some advice on a core architectural question.

We want to fill out cost information on an augmenting (receiving) leg of a transfer, such that it matches the cost information on the reducing legs of the transfer.  Should this be part of:

1) booking, because booking fills out cost information, and in particular, here we are filling out exactly the same cost information on the augmenting legs that booking is computing for the reducing legs,

or

2) imputation,

Fwiw I dub this "interpolation" in the code

because imputation is about inferring values to make the transaction balance, and to make a transaction balance, the augmenting cost info must match the reducing cost info

Conceptually, making it part of imputation feels more correct, but booking is already filling out some cost info (the date) and we'd have to undo that in imputation -- implementation wise it feels easier to implement this in booking.

It's really not obvious. I've thought a lot about them interplay of booking an interpolation in the past and looked at many scenarios and haven't found an elegant solution yet (one may exist). I'm not super happy about the ordering in which I carry out those two. I think what I do now is try to do the reductions first in order to gather information that I can then use in interpolation. There are some comments in the code about this.

On Sat, Aug 19, 2023 at 5:27 PM Eric Altendorf <erical...@gmail.com> wrote:

Well, I actually just realized that I can't assume there is only one reduction posting, because the reduction may pull from multiple lots.  I need to investigate more.

On Sat, Aug 19, 2023 at 2:05 PM Eric Altendorf <erical...@gmail.com> wrote:

Hi all,

A few recent (and older threads) have discussed "transferring" cost information from account to account in a transfer transaction (reduction in one account with a balancing augmentation in another account).

I've studied the code and current behavior and it seems quite feasible to improve how this works.  I wrote up my findings and proposal here:

https://docs.google.com/document/d/1M0W9U5ZdwkiziLNKfiQVIHYSI7gz_gM9pMQhicNXbE0/edit?usp=sharing

I am sure my understanding of the system is limited and flawed, however, so I would greatly appreciate feedback, on my understanding of the problem and the current behavior, as well as on the proposed changes.

Thank you in advance,

eric

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAFXPr0uQH%2BwU37Rj5CWnNRW%3D5%2Bdmxiu7Ccz5nedd3YE2tkJ3OA%40mail.gmail.com.

[Eric Altendorf]

On Tue, Aug 29, 2023 at 5:03 PM Martin Blais <bl...@furius.ca> wrote:

On Sat, Aug 19, 2023, 21:14 Eric Altendorf <erical...@gmail.com> wrote:

So having thought about this issue some more, I think I need some advice on a core architectural question.

We want to fill out cost information on an augmenting (receiving) leg of a transfer, such that it matches the cost information on the reducing legs of the transfer.  Should this be part of:

1) booking, because booking fills out cost information, and in particular, here we are filling out exactly the same cost information on the augmenting legs that booking is computing for the reducing legs,

or

2) imputation,

Fwiw I dub this "interpolation" in the code

Ah yes.  Not sure why I starting saying imputation.

 

because imputation is about inferring values to make the transaction balance, and to make a transaction balance, the augmenting cost info must match the reducing cost info

Conceptually, making it part of imputation feels more correct, but booking is already filling out some cost info (the date) and we'd have to undo that in imputation -- implementation wise it feels easier to implement this in booking.

It's really not obvious. I've thought a lot about them interplay of booking an interpolation in the past and looked at many scenarios and haven't found an elegant solution yet (one may exist). I'm not super happy about the ordering in which I carry out those two. I think what I do now is try to do the reductions first in order to gather information that I can then use in interpolation. There are some comments in the code about this.

Makes sense.  I haven't thought deeply enough about what's really going on here to have a strong intuition about the right way to structure this, although as we discussed, whatever the structure, it seems promising to formulate it in terms of a known CS problem like unification, satisfiability, logical inference, and then farm it out to a generic inference engine.

The little intuition I have is that it could make sense to treat separately (1) filling in facts of the transaction which are deterministically derivable from other facts of the transaction, and (2) higher level annotations that are a function of user choice or some selectable algorithm (like booking matches).  And from that perspective, my intuition would be to first do (1) and then do (2) ... but you have a lot more hands-on experience with where these things depend on each other.

FWIW.... Since I wrote the doc I had decided to push forward with prototyping "anything that works" without necessarily getting full buy-in on the direction via the design doc.  I preferred the idea of transferring the cost info in the interpolation phase, but it looked much easier to do it the booking phase.  I think I have it working with almost trivial (if not pretty) code changes to booking.

The next thing I need to look at, though, is the booking methods and how they're selected per account.  The fact that you want to *sell* FIFO doesn't mean that's how you want to *transfer*.  In fact, if you're talking about crypto in an exchange, the first lots you might want to sell are likely the last lots you want to transfer, since you're probably transferring to a cold wallet with the intent to not sell in the near term.  Ultimately there might be a global optimization problem to do lot selection on transfers to ensure that the lots you want to sell are in the right account at the time of sale -- but that's for the fully general problem and I think I'll be fine with something local/greedy.

On Sat, Aug 19, 2023 at 5:27 PM Eric Altendorf <erical...@gmail.com> wrote:

Well, I actually just realized that I can't assume there is only one reduction posting, because the reduction may pull from multiple lots.  I need to investigate more.

On Sat, Aug 19, 2023 at 2:05 PM Eric Altendorf <erical...@gmail.com> wrote:

Hi all,

A few recent (and older threads) have discussed "transferring" cost information from account to account in a transfer transaction (reduction in one account with a balancing augmentation in another account).

I've studied the code and current behavior and it seems quite feasible to improve how this works.  I wrote up my findings and proposal here:

https://docs.google.com/document/d/1M0W9U5ZdwkiziLNKfiQVIHYSI7gz_gM9pMQhicNXbE0/edit?usp=sharing

I am sure my understanding of the system is limited and flawed, however, so I would greatly appreciate feedback, on my understanding of the problem and the current behavior, as well as on the proposed changes.

Thank you in advance,

eric

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAFXPr0uQH%2BwU37Rj5CWnNRW%3D5%2Bdmxiu7Ccz5nedd3YE2tkJ3OA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAK21%2BhM_kkGq8R-ASb%2BVkX5ZoLCdJLp41q43Mi0P3PQXtmEXLA%40mail.gmail.com.