Fixtures teardown should happen in reverse order of setup

38 views
Skip to first unread message

Benjamin BEAUD

unread,
Sep 28, 2023, 10:28:41 AM9/28/23
to lemoncheesecake
Hi, I'm facing an issue with the fixtures teardown. From what I understood from the framework, pairs of functions for setup and teardown are built for each fixtures.
An orderedSet is used to compute dependencies from the fixtures parameters.
Then the setup functions are called following this specific dependency order.
However in the teardown, this same order is used which seems to be inaccurate. I wrote an example here (in test_runner.py file) to highlight the problem.

```python
def test_run_with_fixtures_using_yield_and_dependencies_inverted_teardown():
    """Fixtures teardown must happen in reverse order, even if yield is used for safe teardowns"""
    marker = []

    @lcc.fixture(scope="pre_run")
    def session_fixture_prerun():
        """Some settings"""
        retval = 1
        marker.append(retval)
        yield retval
        marker.append(1)

    @lcc.fixture(scope="session")
    def session_fixture_execute_a(session_fixture_prerun):
        """An admin object builder necessary to manipulate `b` objects"""
        @contextmanager
        def func_a():
            try:
                lcc.log_info("session_fixture_execute_a_setup")
                retval = session_fixture_prerun * 2
                marker.append(retval)
                yield retval
            finally:
                marker.append(2)
                lcc.log_info("session_fixture_execute_a_teardown")
        return func_a

    @lcc.fixture(scope="session")
    def session_fixture_a(session_fixture_execute_a):
        """An admin object instance `a` shared across many fixtures"""
        with session_fixture_execute_a() as a:
            try:
                lcc.log_info("session_fixture_a_setup")
                retval = a * 11
                marker.append(retval)
                yield retval
            finally:
                marker.append(3)
                lcc.log_info("session_fixture_a_teardown")

    @lcc.fixture(scope="session")
    def session_fixture_execute_b(session_fixture_a):
        """A resource object builder manipulated by an admin `a` object"""
        @contextmanager
        def func_b(param):
            try:
                lcc.log_info("session_fixture_execute_b_setup")
                retval = session_fixture_a * 3 * param
                marker.append(retval)
                yield retval
            finally:
                marker.append(4)
                lcc.log_info("session_fixture_execute_b_teardown")
        return func_b

    @lcc.fixture(scope="session")
    def session_fixture_b(session_fixture_execute_b):
        """A resource object instance `b`"""
        with session_fixture_execute_b(17) as b:
            try:
                lcc.log_info("session_fixture_b_setup")
                retval = b * 2
                marker.append(retval)
                yield retval
            finally:
                marker.append(5)
                lcc.log_info("session_fixture_b_teardown")

    @lcc.suite("MySuite")
    class MySuite:
        @lcc.test("Test")
        def test(self, session_fixture_b, ):
            marker.append(session_fixture_b * 6)

    report = run_suite_class(MySuite, fixtures=(session_fixture_prerun, session_fixture_execute_a, session_fixture_a, session_fixture_execute_b, session_fixture_b))

    # test that each fixture value is passed to test or fixture requiring the fixture
    assert marker == [1, 2, 22, 1122, 2244, 13464, 5, 4, 3, 2, 1]
    # Current value is [1, 2, 22, 1122, 2244, 13464, 3, 2, 5, 4, 1], fixtures `a` are teardown before fixtures `b`

    # check that each fixture setup and teardown is properly executed in the right order
    assert report.test_session_setup.get_steps()[0].get_logs()[0].message == "session_fixture_execute_a_setup"
    assert report.test_session_setup.get_steps()[0].get_logs()[1].message == "session_fixture_a_setup"
    assert report.test_session_setup.get_steps()[0].get_logs()[2].message == "session_fixture_execute_b_setup"
    assert report.test_session_setup.get_steps()[0].get_logs()[3].message == "session_fixture_b_setup"
    assert report.test_session_teardown.get_steps()[0].get_logs()[0].message == "session_fixture_b_teardown"
    assert report.test_session_teardown.get_steps()[0].get_logs()[1].message == "session_fixture_execute_b_teardown"
    assert report.test_session_teardown.get_steps()[0].get_logs()[2].message == "session_fixture_a_teardown"
    assert report.test_session_teardown.get_steps()[0].get_logs()[3].message == "session_fixture_execute_a_teardown"
```

I suggest the following fix in runner.py:

```python
    def run_teardown_funcs(self, teardown_funcs):
        for teardown_func in reversed(teardown_funcs):
            if teardown_func:
                try:
                    teardown_func()
                except Exception as e:
                    self.handle_exception(e)
```

Also pytest is quite explicit on this point:
https://docs.pytest.org/en/7.1.x/how-to/fixtures.html#yield-fixtures-recommended
`Because receiving_user is the last fixture to run during setup, it's the first to run during teardown.`

Nicolas Delon

unread,
Oct 2, 2023, 6:00:42 PM10/2/23
to lemoncheesecake
Hello Benjamin,

You're right, teardown-ing same-scope fixtures in the LIFO order makes more sense. I just released a 1.14.3 version that implements that change.

Thanks for pointing this out.

Best regards,

Nicolas.

NB: for eventual new feedback, you can fill a ticket on github (https://github.com/lemoncheesecake/lemoncheesecake/issues/new) which is more convenient to handle.

Benjamin BEAUD

unread,
Oct 5, 2023, 9:22:06 AM10/5/23
to lemoncheesecake
Thanks !
Reply all
Reply to author
Forward
0 new messages