ofx max_date correct?

[Frederick Noon]

Advice requested...

The off-the-shelf v2 ingest/importers/ofx.py uses the DTASOF value of the LEDGERBAL section as the file date.  However, I find that this date is consistently the date I downloaded the data from my bank, not the closing date of the period I asked for.  So if, in April, I download transactions for January into one file, then download transactions for February into another, I get a name collision when running bean-file as ofx.py says each file has a max_date in April.

To change this behavior in my installation I've modified ofx.py to look for the DTEND value in the BANKTRANLIST section, which gives me the end of the period I queried my bank for.  This has worked out very well for me.

So: is this a bug in ofx.py, or am I misapplying the importers feature?  Or is this a behavior that varies widely from bank to bank?  Should I pack my changes into a pull request for others to use?

Thanks in advance.

[Martin Blais]

These importers are intended merely as examples and aren't supported as general importers.

The OFX importer, in particular, is based on a particular use case and is not a general purpose thing that would work in every case.

I suggest you use one of the OFX libraries on github to build yourself an importer that will support the tags your particular institution yields. (Unfortunately, there's a lot of variation between institutions, even within the format.)

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/5d9554cb-d2a7-4dbe-94d7-1e872173b85an%40googlegroups.com.

[Frederick Noon]

Will-do.  I found that my change matched the usage of at least 3 of the institutions I was using, but didn't know how representative my sample was.

[Martin Michlmayr]

I also noticed that balance statements have the wrong date with some
(but not all) banks when using the beancount OFX, which matches
exactly what you describe.

* Frederick Noon <frs....@gmail.com> [2020-10-21 11:38]:

> Will-do. I found that my change matched the usage of at least 3 of the
> institutions I was using, but didn't know how representative my sample was.
>
> On Wednesday, October 21, 2020 at 7:13:28 AM UTC-7 bl...@furius.ca wrote:
>
> > These importers are intended merely as examples and aren't supported as
> > general importers.
> > The OFX importer, in particular, is based on a particular use case and is
> > not a general purpose thing that would work in every case.
> >
> > I suggest you use one of the OFX libraries on github to build yourself an
> > importer that will support the tags your particular institution yields.
> > (Unfortunately, there's a lot of variation between institutions, even
> > within the format.)
> >
> >
> >
> > On Tue, Oct 20, 2020 at 8:50 PM Frederick Noon <frs....@gmail.com> wrote:
> >
> >> Advice requested...
> >>

> >> The off-the-shelf v2 *ingest/importers/ofx.py* uses the *DTASOF* value
> >> of the *LEDGERBAL* section as the file date. However, I find that this

> >> date is consistently the date I downloaded the data from my bank, not the
> >> closing date of the period I asked for. So if, in April, I download
> >> transactions for January into one file, then download transactions for

> >> February into another, I get a name collision when running *bean-file*
> >> as *ofx.py* says each file has a max_date in April.
> >>
> >> To change this behavior in my installation I've modified *ofx.py* to
> >> look for the *DTEND* value in the *BANKTRANLIST* section, which gives me

> >> the end of the period I queried my bank for. This has worked out very well
> >> for me.
> >>

> >> So: is this a bug in *ofx.py,* or am I misapplying the importers

> >> feature? Or is this a behavior that varies widely from bank to bank?
> >> Should I pack my changes into a pull request for others to use?
> >>
> >> Thanks in advance.
> >>
> >> --
> >> You received this message because you are subscribed to the Google Groups
> >> "Beancount" group.
> >> To unsubscribe from this group and stop receiving emails from it, send an
> >> email to beancount+...@googlegroups.com.
> >> To view this discussion on the web visit
> >> https://groups.google.com/d/msgid/beancount/5d9554cb-d2a7-4dbe-94d7-1e872173b85an%40googlegroups.com

> >> <https://groups.google.com/d/msgid/beancount/5d9554cb-d2a7-4dbe-94d7-1e872173b85an%40googlegroups.com?utm_medium=email&utm_source=footer>
> >> .

> >>
> >
>
> --
> You received this message because you are subscribed to the Google Groups "Beancount" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/36f21b62-70c9-4c23-a9b3-1f3de2bb83e5n%40googlegroups.com.


--
Martin Michlmayr
https://www.cyrius.com/

[Red S]

Ditto, banks don't seem to be fully consistent on these.

FWIW, for my importers, I use the following dates via ofxparse:

• date of last transaction, used for filing via bean-file: I use ofxparse's statement.end_date, which in turn is from DTEND
• balance assertions date: ofxparse uses DTASOF. However, I find this to be a problem for the reason here. So I computed it as the date of the last transaction + 1
  
• commodity price date: this is typically priced on the download date. ofxparse uses DTASOF, I believe.
  

[TRS-80]

In case you guys were not aware, there is an effort to compare
notes and experiences (including a forum), as well as a database
of connection parameters over at OFXHome.[0]

In fact many of the various OFX projects actually pull their
particular connection information from there.

But my understanding is all of these valuable little bits of
information have essentially been reverse engineered and shared
by people like us just trying to figure things out.

It's an "open" protocol, albeit with little to no documentation
from the banks themselves. So it's up to us to help each other
out. It is in that spirit that I encourage anyone who is hacking
on OFX stuff to please consider contributing to the knowledge
base there. Forums are slow, but I still found some very helpful
information there from time to time.

Cheers!

TRS-80

[0] https://www.ofxhome.com