Importer.extract() raised an unexpected error: can't compare datetime.datetime to datetime.date

[Marcio A. Vianna F. (mvianna)]

Hi,

I'm taking up beancount again after some 2 years away from it. My biggest difficulty had then been importing bank and brokers' statements to beancount. I was getting the best results with (modified) reds_importers + smart_importer, which came to work reasonably well. But I'm realizing that retaking from where I left off will not be as simple as I wished.

I'm getting the error in the subject line. I suppose this error was introduced for two reasons (please confirm):

1) Martin, at some point, agreed to include, not only date, but also time of transactions, according to requests from users interested in daytrading (which I am too, and welcomed his decision). Martin didn't see then any reason why this would cause inconsistencies in beancount. I don't remember it causing any issues with reds_importers, fava, smart_importer either.

2) Some later python release made the comparison between datetime.date and datetime.datetime incompatible. This comparison is needed in the chronological ordering of transactions.

I first got this error in reds_importers and think I managed to fix it by making the simple change in ofxreader.py:

from:

            date = max(ot.tradeDate if hasattr(ot, 'tradeDate') else ot.date

                       for ot in self.get_transactions()).date()  

to:

            date = max(ot.tradeDate if hasattr(ot, 'tradeDate') else ot.date

                       for ot in self.get_transactions())     #.date()

But then I started getting the same error message from smart_importer, which makes me think I should be better to avoid the python version update that made date and datetime incomparable.

Any suggestions? Which python version is the latest that doesn't cause this kind of error? Has anyone else encountered these issues? I searched the forum and the internet and didn't find anything.

Thanks

Marcio

[Red S]

Hello there,
A few observations:

1. AFAIK, Beancount doesn’t support timestamps. Please correct me if this is out of date.

2. With the change you made to ofxreader, it ends up emitting a datetime instead of a date, which would break with anything downstream that expects a date (and correctly so). Instead of making this change, you'd have to root cause the issue and resolve it correctly. If you need help, feel free to post a test case and a stack trace and I'll take a look when I'm able

3. Are you using the latest, (unreleased) commit with beancount_reds_importers? If not, please try that out first, it fixes a related issue

​

[Marcio A. Vianna F. (mvianna)]

Many thanks for your response. My comments are below:

1. 
   AFAIK, Beancount doesn’t support timestamps. Please correct me if this is out of date.

I think you are right. What Martin proposed in this posting ("Here's what I could do for you...") was only a workaround. He would include the possibility of timestamps in the parser only. However, the timestamp part wouldn't be used downstream in any way. The timestamp would be recorded in a custom field and any further processing of its value would be left to the user possibly through plugins. I read somewhere else, though, that the full timestamp (date+time) is to be implemented in v.3.

1. With the change you made to ofxreader, it ends up emitting a datetime instead of a date, which would break with anything downstream that expects a date (and correctly so). Instead of making this change, you'd have to root cause the issue and resolve it correctly. If you need help, feel free to post a test case and a stack trace and I'll take a look when I'm able

The funny thing is that the error message actually disappeared (from reds_importers modules) after I made the change (and re-emerged in smart_importer modules). I thought: if instead of a date, I input a datetime in the beancount file, python won't complain anymore, since it will be comparing datetime to datetime now. I am by no means a developer or hacker and that was just a guess. I'll try to update to the unreleased version, as you suggest below and see if the issue gets solved. Do you mean the latest unreleased version of beancount (v.2) or beancount_reds_importers? Could you point me to how to install an unreleased version? Is it the pip install +git... method?

Regards

Marcio

[Red S]

The funny thing is that the error message actually disappeared (from reds_importers modules) after I made the change (and re-emerged in smart_importer modules). I thought: if instead of a date, I input a datetime in the beancount file, python won't complain anymore, since it will be comparing datetime to datetime now. I am by no means a developer or hacker and that was just a guess.

Yes, it’s actually a common pattern for the wrong fix to seemingly make an error message disappear, but in reality, the true issue gets suppressed, only to manifest elsewhere.

I'll try to update to the unreleased version, as you suggest below and see if the issue gets solved. Do you mean the latest unreleased version of beancount (v.2) or beancount_reds_importers? Could you point me to how to install an unreleased version? Is it the pip install +git... method?

beancount_reds_importers. From README:

pip3 install git+https://github.com/redstreet/beancount_reds_importers

​