Test Issues

[Michael Robellard]

So I got around to looking at stuff today and cloned the yosai github repo and ran into some issues:

I can't seem to get the tests to run. I am enclosing the output of the tests. I am guessing there is either something missing in my virtual env or there is some configuration missing.

[Stefane Fermigier]

I also have problems with the tests (for instance, tox fails because of a superfluous '-rrequirements.txt' in the deps section).

At least you've succeeded at running them. How did you do ?

Running 'py.test test' on the root directory I get:

yosai.core.exceptions.FileNotFoundException: could not locate: yosai_settings.yaml

ERROR: could not load /Users/fermigier/projects/yosai/test/conftest.py

Regards,

  S.

[Michael Robellard]

I just ran py.test in the project root but I have a yosai_settings.yaml file in the root, I have enclosed my copy of the file, I am not planning on using this secret key for anything other than testing.

[Stefane Fermigier]

Thanks. Lots on errors also on my box (1 failed, 538 passed, 1 xfailed, 1 xpassed, 178 error in 6.11 seconds).

Also, I've had to create a /var/log/yosai/ directory for the tests to run.

  S.

[Michael Robellard]

That's the same output I got, so we are at the same place. We just need to know what we are missing

[Darin Gordon]

Tests are categorized as integrated and isolated (unit).  

Integrated tests are used for complete, end-to-end tests.  Consequently, integrated tests require a cache and accountstore.  For integrated testing, YosaiAlchemyStore is used with a sqlite backend and YosaiDPCache with a redis backend.  

Isolated (unit) tests are independent tests which use a mix of classic and mock testing.  Neither extension project is needed to run them.

I updated the README.md for each extension project (YosaiAlchemyStore and YosaiDPCache) explaining how to configure the project.  Both projects follow a similar convention.

To run integrated tests, use the env-config-file approach, defining system environment variables that reference the locations of the yaml files required for yosai, yosai_alchemystore, and yosai_dpcache and ensure that file readable permissions:

YOSAI_CORE_SETTINGS=/path/to/your/yosai_settings.yaml

YOSAI_CACHE_SETTINGS=/path/to/your/cache_settings.yaml

YOSAI_ALCHEMYSTORE_SETTINGS=/path/to/your/accountstore_settings.yaml

Again, Yosai uses sqlite as the backend for AlchemyAccountStore for integrated testing.  The accountstore_settings.yaml is simply:

<--------------- cut below this line

---

ENGINE_CONFIG: 

    dialect:  sqlite

    path: '//'

    userid:

    password:

    hostname:

    port:

    db:

---------------> cut above this line

Hopefully, this covers everything.  I'm sure I'll know soon enough.

[Michael Robellard]

Starting to make some progress here.

After getting my configuration files fixed

my python cryptography library fixed (There are issues in MacOS X with the python cryptography library that requires special install steps, see https://cryptography.io/en/latest/installation/#building-cryptography-on-os-x for details)

Installed missing requirements (pytest and pytest-capturelog are required to run the tests, There is a pull request to update the documentation)

I now have no errors on running the tests and 20 failures. Enclosed is the output from my run of py.test. Hopefully we can get the tests running cleanly so I know that I have a good base to work from.

Just a thought, it might be a good idea to look into using Travis CI to run the tests automatically since it is free for open source projects and integrates nicely with github.

[Darin Gordon]

Are you volunteering to bring it altogether?  I support the idea.  

[Michael Robellard]

Darin,

I think once we get the failures figured out (so that I know the test are working before I try and run them in another environment) I could probably set up Travis (I haven't done it myself yet, but I have been wanting to try it.) It look pretty straightforward. Since you are the project admin you have to go to the travis-ci website and enable travis for the project, and then we have to build a travis.yml file to configure travis, which I would be willing to take a stab at.

[Michael Robellard]

Looking at the failures, It looks like caplog.text is a function in pytest-captuelog, but it has always been a function so this should never have worked, I am wondering if you are using something else for the capturelog stuff that would be causing the issue? All 20 failures seem to be the same thing.

On Monday, April 18, 2016 at 10:17:11 AM UTC-4, Darin Gordon wrote:

[Michael Robellard]

Everything is passing now. The last 20 failures were being caused by an incorrect module. The required module should be pytest-catchlog not pytest-capturelog. I made a pull request to update the documentation. Now I feel comfortable that I can move forward with trying to use the software. I will also start working on the travis ci yaml file.