How to use pytests for personal importers / ingest

[Vivek Gani]

Hi, I'm trying to wrap my head around how writing importers (and their tests) have changed since the migration from using nosetests to pytest (which occurred back in Aug 2018  - https://bitbucket.org/blais/beancount/commits/758c07ae150c0e3a30c77421880508664ffc0965 ). Particularly for the usecase of personal importers kept in a ./importers directory relative to where they keep their beancount ledger.

Assume one has a directory laid out as:

• myBeancount/
• myBeancount.beancount
  
• importers/
• __init__.py (empty)
  
• conftest.py
  
• utrade/
• __init__.py (empty)
  
• utrade_csv.py
• utrade_csv_test.py
• UTrade20140713.csv
  

A few points from this:

•  "all regression files must have all accompanying expect files and all must pass" - Okay so the .extract, .file_account, .file_date, .file_name need to already be present (and presumably created by hand before), and one can do `pytest --generate` to create the files if needed (done via beancount/ingest/regression_pytest.py)
  
  
• In order to do `pytest --generate` with your own importers directory, you need to similarly have a conftest.py in your importers directory:
  
  # This adds the --generate option.
  # pylint: disable=invalid-name
  pytest_plugins = "beancount.ingest.regression_pytest"
  
  
  You should be able to leave your importer's __init__.py  as-is (usually empty).
  
  
• The `.pytest_cache` directory in beancount is in the beancount top-level directory, and should not exist in subdirectories. This is where I'm getting lost. If I have my own importers directory, is there a settings file I need to make or something to avoid having a `.pytest_cache` get created by default in subdirectories?
  
  
• For example, if I try to run pytest within the UTrade directory (within beancount), things work fine. However when trying to do it within my example (myBeancoung/importers/utrade), it seems to end up choking on it's a `.pytest_cache` directory that gets created in the UTrade directory, leading to errors like this:
  
  
  E           AssertionError: Expected file '/Users/me/example_beancount/importers/utrade/.pytest_cache/.gitignore.file_name' is missing. Generate it?
  E           assert False
  E            +  where False = <function exists at 0x109713048>('/Users/me/example_beancount/importers/utrade/.pytest_cache/.gitignore.file_name')
  E            +    where <function exists at 0x109713048> = path.exists
  
  
  
  
• If I try to run pytest from just the myBeancount directory, I get an error about the non-top-level conftest file.
  Notably, I get this same error in beancount itself if I try to do 'make vtest' (which tries to test examples):
  
  
  ___________________________________________________________________ ERROR collecting test session ____________________________________________________________________
  Defining 'pytest_plugins' in a non-top-level conftest is no longer supported:
  It affects the entire test suite instead of just below the conftest as expected.
    /Users/me/beancount/examples/ingest/office/importers/conftest.py
  Please move it to a top level conftest file at the rootdir:
    None
  For more information, visit:
    https://docs.pytest.org/en/latest/deprecations.html#pytest-plugins-in-non-top-level-conftest-files
  
  
  (see also an earlier thread on importers / tests "Nose tests for ingest - do they work OOTB?": https://groups.google.com/forum/#!searchin/beancount/pytest%7Csort:date/beancount/6D2VEdpsWJc/_7EsVzmLBAAJ )
  

[Vivek Gani]

Okay, realized for the case of one's personal importers you can solve the test issues just by keeping conftest.py file in the myBeancount directory.

• myBeancount/
• myBeancount.beancount
• conftest.py
• importers/
• __init__.py (empty)
  

• utrade/
• __init__.py (empty)
  
• utrade_csv.py
• utrade_csv_test.py
• UTrade20140713.csv