Lost cost?

[Eric Altendorf]

I've got my first importer running, mostly works, but the cost I'm providing doesn't seem to be making it all the way through.  A debug print shows this as the transaction I'm returning:

Transaction(meta={'filename': '/home/...csv', 'lineno': 1}, date=datetime.date(2018, 1, 1), flag='*', payee=None, narration='Bought 2.0 ABC for $204.00 USD', tags=frozenset(), links={None}, postings=[Posting(account='Assets:MyInstitution:ABC', units=2.0 ABC, cost=Cost(number='102.00', currency='USD', date=None, label=None), price=102.00 USD, flag=None, meta=None), Posting(account='Assets:MyInstitution:USD', units=-204.00 USD, cost=None, price=None, flag=None, meta=None)])

But the final transaction printed by the importer has no cost:

2018-01-01 * "Bought 2.0 ABC for $204.00 USD" ^None
  Assets:MyInstitution:ABC          2.0 ABC {} @ 102.00 USD
  Assets:MyInstitution:USD      -204.00 USD

Any ideas?

(PS: Also not sure how to omit links.  If I return `None`, `""`, or `{}`, the importer crashes.  If I return `{None}` it complete but renders the annoying ^None link....)

[Daniele Nicolodi]

On 18/07/23 04:47, Eric Altendorf wrote:
> I've got my first importer running, mostly works, but the cost I'm
> providing doesn't seem to be making it all the way through.  A debug
> print shows this as the transaction I'm returning:
>
> Transaction(meta={'filename': '/home/...csv', 'lineno': 1},
> date=datetime.date(2018, 1, 1), flag='*', payee=None, narration='Bought
> 2.0 ABC for $204.00 USD', tags=frozenset(), links={None},
> postings=[Posting(account='Assets:MyInstitution:ABC', units=2.0 ABC,

> *cost=Cost(number='102.00', currency='USD', date=None, label=None)*,

> price=102.00 USD, flag=None, meta=None),
> Posting(account='Assets:MyInstitution:USD', units=-204.00 USD,
> cost=None, price=None, flag=None, meta=None)])
>
> But the final transaction printed by the importer has no cost:
>
> 2018-01-01 * "Bought 2.0 ABC for $204.00 USD" ^None

>   Assets:MyInstitution:ABC          2.0 ABC*{} *@ 102.00 USD

>   Assets:MyInstitution:USD      -204.00 USD
>
> Any ideas?

The number member in the Cost class should be a Decimal, not a string.
The entry printer should probably error out if this is not the case, but
it does not.

> (PS: Also not sure how to omit links.  If I return `None`, `""`, or
> `{}`, the importer crashes.  If I return `{None}` it complete but
> renders the annoying ^None link....)

links and tags are sets. If you don't want any links or tags just assign
an empty set. Somehow you figured this out for tags but not for links.

One way to discover the data structure corresponding to a given
transaction is to write its text representation and parse it with the
beancount parser:

from beancount.parser import parser

entries, errors, options = parser.parse_string("""

2018-01-01 * "Bought 2.0 ABC for $204.00 USD"

Assets:MyInstitution:ABC 2.0 ABC {102.00 USD} @ 102.00 USD

Assets:MyInstitution:USD -204.00 USD

""")

print(entries)


Cheers,
Dan

[Eric Altendorf]

On Tue, Jul 18, 2023 at 9:57 AM Daniele Nicolodi <dan...@grinta.net> wrote:

On 18/07/23 04:47, Eric Altendorf wrote:
> I've got my first importer running, mostly works, but the cost I'm
> providing doesn't seem to be making it all the way through.  A debug
> print shows this as the transaction I'm returning:
>
> Transaction(meta={'filename': '/home/...csv', 'lineno': 1},
> date=datetime.date(2018, 1, 1), flag='*', payee=None, narration='Bought
> 2.0 ABC for $204.00 USD', tags=frozenset(), links={None},
> postings=[Posting(account='Assets:MyInstitution:ABC', units=2.0 ABC,
> *cost=Cost(number='102.00', currency='USD', date=None, label=None)*,
> price=102.00 USD, flag=None, meta=None),
> Posting(account='Assets:MyInstitution:USD', units=-204.00 USD,
> cost=None, price=None, flag=None, meta=None)])
>
> But the final transaction printed by the importer has no cost:
>
> 2018-01-01 * "Bought 2.0 ABC for $204.00 USD" ^None
>    Assets:MyInstitution:ABC          2.0 ABC*{} *@ 102.00 USD
>    Assets:MyInstitution:USD      -204.00 USD
>
> Any ideas?

The number member in the Cost class should be a Decimal, not a string.
The entry printer should probably error out if this is not the case, but
it does not.

Nice catch, thanks.

I should figure out how to get mypy working on this code; this is the second or

third time a simple type error has bitten me.

> (PS: Also not sure how to omit links.  If I return `None`, `""`, or
> `{}`, the importer crashes.  If I return `{None}` it complete but
> renders the annoying ^None link....)

links and tags are sets. If you don't want any links or tags just assign
an empty set. Somehow you figured this out for tags but not for links.

Ooof.  {} is not an empty set, it's an empty associative array.  Python

is not my strongest (or favorite) language.  The code I was working

from used data.EMPTY_SET for the tags and I didn't put two and two

together and realize that's what I should use for tags too.  (Or `set()`)

One way to discover the data structure corresponding to a given
transaction is to write its text representation and parse it with the
beancount parser:

from beancount.parser import parser

entries, errors, options = parser.parse_string("""
2018-01-01 * "Bought 2.0 ABC for $204.00 USD"
   Assets:MyInstitution:ABC      2.0 ABC {102.00 USD} @ 102.00 USD
   Assets:MyInstitution:USD  -204.00 USD
""")

print(entries)

Great recipe, thanks, this will be handy.

 

Cheers,
Dan

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/5385377a-06a9-be25-50d3-d2d88fd8329f%40grinta.net.

[Eric Altendorf]

On Tue, Jul 18, 2023 at 12:31 PM Eric Altendorf <erical...@gmail.com> wrote:

On Tue, Jul 18, 2023 at 9:57 AM Daniele Nicolodi <dan...@grinta.net> wrote:

On 18/07/23 04:47, Eric Altendorf wrote:
> I've got my first importer running, mostly works, but the cost I'm
> providing doesn't seem to be making it all the way through.  A debug
> print shows this as the transaction I'm returning:
>
> Transaction(meta={'filename': '/home/...csv', 'lineno': 1},
> date=datetime.date(2018, 1, 1), flag='*', payee=None, narration='Bought
> 2.0 ABC for $204.00 USD', tags=frozenset(), links={None},
> postings=[Posting(account='Assets:MyInstitution:ABC', units=2.0 ABC,
> *cost=Cost(number='102.00', currency='USD', date=None, label=None)*,
> price=102.00 USD, flag=None, meta=None),
> Posting(account='Assets:MyInstitution:USD', units=-204.00 USD,
> cost=None, price=None, flag=None, meta=None)])
>
> But the final transaction printed by the importer has no cost:
>
> 2018-01-01 * "Bought 2.0 ABC for $204.00 USD" ^None
>    Assets:MyInstitution:ABC          2.0 ABC*{} *@ 102.00 USD
>    Assets:MyInstitution:USD      -204.00 USD
>
> Any ideas?

The number member in the Cost class should be a Decimal, not a string.
The entry printer should probably error out if this is not the case, but
it does not.

Do you know where in the code this was getting checked and dropped?

I'm encountering a similar problem with trying to set a {USD} marker

CostSpec, as the workaround for transferring cost basis through account

tranfsers, as described here:

https://github.com/beancount/beancount/issues/476

I've updated my importer to add such a CostSpec to transfer postings,

and it appears in the transactions I create:

Transaction(meta={'filename': '/home/eric/crypto-taxes/beancount-crypto-importers/tests/Coinbase-d34db33f-TransactionsHistoryReport-2020-01-01-00-00-00.csv', 'lineno': 1}, date=datetime.date(2020, 2, 20), flag='*', payee=None, narration='CB: Sent 0.01BTC to adh547SDFhs7SDKJFkdfh4KS95hsdkfs6s', tags=frozenset(), links=set(), postings=[Posting(account='Assets:Coinbase:BTC', units=-0.01 BTC, cost=CostSpec(number_per=None, number_total=None, currency='USD', date=None, label=None, merge=None), price=None, flag=None, meta=None), Posting(account='Assets:ALLEXTERNAL:BTC', units=0.01 BTC, cost=CostSpec(number_per=None, number_total=None, currency='USD', date=None, label=None, merge=None), price=None, flag=None, meta=None)])

However in the final output from the importer, it's missing:

2020-02-20 * "CB: Sent 0.01BTC to adh547SDFhs7SDKJFkdfh4KS95hsdkfs6s"
  Assets:Coinbase:BTC     -0.01 BTC {}
  Assets:ALLEXTERNAL:BTC   0.01 BTC {}

Test input/output:

  https://github.com/ericaltendorf/beancount-crypto-importers/tree/master/tests

Code: 

  https://github.com/ericaltendorf/beancount-crypto-importers/blob/master/importer-coinbase.py#L118

  https://github.com/ericaltendorf/beancount-crypto-importers/blob/master/common.py

thanks,

eric

[Daniele Nicolodi]

On 19/07/23 19:11, Eric Altendorf wrote:
> Do you know where in the code this was getting checked and dropped?

It should be here
https://github.com/beancount/beancount/blob/193645460fd7aafcb3d9e0359edabc21e389db81/beancount/core/position.py#L64-L104

> I'm encountering a similar problem with trying to set a {USD} marker
> CostSpec, as the workaround for transferring cost basis through account
> tranfsers, as described here:
> https://github.com/beancount/beancount/issues/476

Reading the code above, serializing a CostSpec with a currency but no
unit or total cost does not seem to be supported. If there is an use
case for this, the serialization function needs to be patched.

Cheers,
Dan

[Eric Altendorf]

Excellent, thank you!  I can patch this locally so I can make progress.

I know Martin is busy this week, but after he's back, I'll leave it up to him

whether it makes sense to do this patch in general, and/or to address

https://github.com/beancount/beancount/issues/481

Either way, silent dropping of data in serialization seems like something

that might be good to address -- if there are self-consistency rules for the

Position data structure, we might want to enforce those at construction

time, as you suggested.

 

Cheers,
Dan

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/430dc956-7186-d485-52a2-68edf3f1f56e%40grinta.net.