Regression test best/common practices

[David Richey]

Hello,

I'm trying to set up regression tests for my importers, and I'm wondering about how people tend to do this (beyond the basic setup described in the docs).

Specifically, would it be wrong to want to regression test all of my imported documents? If not, do you manually copy all of them from the "documents" directory to the "importers" directory? That feels bad. I fiddled around with symlinking the former into the latter, but I couldn't see any functionality in regression_pytest that would read the sample files from one directory and output the .extract, .file_account, etc. files to a different one (since I don't want to pollute my documents dir).

Does anyone have any suggestions/thoughts on this? Either in my current direction or moving towards not testing against all inputs (although this initially makes me uneasy, maybe you can explain why it shouldn't). 

Regards,

David

[Martin Blais]

Short answer: Use beangulp.

Each importer can be invoked directly, with "identify", "extract", "archive" commands, but also "test" and "generate" commands. 

Write a driver script that will find all of the files you'd like to test and invoke your importer implementation (just the one .py file) on each of those files with the test command.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/91792c68-aaf1-4479-89fc-fbcd7dbf0c9fn%40googlegroups.com.

[David Richey]

I'm not entirely sure if I'm able to use beangulp since I'm using beancount v2. Everything I've read indicates that beancount v3 is not ready for use.

Even so, based on my understanding of the two tools (beangulp and the v2 import tooling), it's not clear to me how beangulp improves on the old tooling in this area. Specifically, it's still not clear to me how to run the regression tests against all of my documents in a nice way. You mentioned writing a script, but would this script look basically the same between the two? 

[Martin Blais]

On Mon, Jan 10, 2022 at 12:54 AM David Richey <dari...@gmail.com> wrote:

I'm not entirely sure if I'm able to use beangulp since I'm using beancount v2. Everything I've read indicates that beancount v3 is not ready for use.

Create a second setup with v3 just to run beangulp.

v3 is working today (that's all I use), but there's not a whole lot of newness turned on yet.

 

Even so, based on my understanding of the two tools (beangulp and the v2 import tooling), it's not clear to me how beangulp improves on the old tooling in this area. Specifically, it's still not clear to me how to run the regression tests against all of my documents in a nice way. You mentioned writing a script, but would this script look basically the same between the two? 

https://docs.google.com/document/d/1O42HgYQBQEna6YpobTqszSgTGnbRX7RdjmzR2xumfjs/ 

See "Test Subcommand" section.

 

 

On Saturday, January 8, 2022 at 4:40:17 PM UTC-6 bl...@furius.ca wrote:

Short answer: Use beangulp.

Each importer can be invoked directly, with "identify", "extract", "archive" commands, but also "test" and "generate" commands. 

Write a driver script that will find all of the files you'd like to test and invoke your importer implementation (just the one .py file) on each of those files with the test command.

On Sat, Jan 8, 2022 at 3:15 PM David Richey <dari...@gmail.com> wrote:

Hello,

I'm trying to set up regression tests for my importers, and I'm wondering about how people tend to do this (beyond the basic setup described in the docs).

Specifically, would it be wrong to want to regression test all of my imported documents? If not, do you manually copy all of them from the "documents" directory to the "importers" directory? That feels bad. I fiddled around with symlinking the former into the latter, but I couldn't see any functionality in regression_pytest that would read the sample files from one directory and output the .extract, .file_account, etc. files to a different one (since I don't want to pollute my documents dir).

Does anyone have any suggestions/thoughts on this? Either in my current direction or moving towards not testing against all inputs (although this initially makes me uneasy, maybe you can explain why it shouldn't). 

Regards,

David

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/91792c68-aaf1-4479-89fc-fbcd7dbf0c9fn%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/dc23e215-4403-499b-9d3e-244035019e1bn%40googlegroups.com.

[David Richey]

Please correct me if I'm wrong, but I don't think I can do that and still use fava and smart_importer. 

[Alan H]

I've been testing my importers using :

```

from beancount.ingest import regression_pytest as regtest
```

And then letting pytest --generate the support files from the given input PDFs (in my case).

For plugins I've been testing using the doc string transaction support available in  `beancount.parser.cmptest`  and `beancount.loader` .

I'm not intending to hijack the thread  - but is there any forecasted timeline for getting fava working with v3? What's the t-shirt size effort required (having not looked, I realise this is a slightly silly question).

Alan