Marked Async Tests Are Skipped

[pattisdr]

Bug Description

Related to #1174

We recently made a lot of our tests async, and pytest is skipping all of them. The async tests are some of our more important tests that are often testing parts of the graph traversal or an end-to-end privacy request.

Steps to Reproduce

Run any async test and see that pytest skips.

Expected behavior

I think we can decorate all these tests with @pytest.mark.asyncio and they should run.

Screenshots

If applicable, add screenshots to help explain your problem.

Environment

• Version:
• OS:
• Python Version:
• Docker Version:

Additional context

Add any other context about the problem here.

[sanders41]

You used to have to mark with @pytest.mark.asyncio, but the pytest-asyncio plugin made a change and doesn't recommend that any more. Are the async tests that are getting skipped defined as async? async def test_...?

When I tested initially I made sure none were skipped. Can you point me to somewhere it is skipping and I will look into it?

[sanders41]

I think I found what you are talking about. It seems to always be marked tests. I wonder if that is throwing something off. We could try the async decorator to see if it makes a difference.

[pattisdr]

Yes, I had just noticed an async def test was getting skipped locally, checked main and realized they were all EDIT more were being skipped - the async decorator worked for the particular test I examined. EDIT: I see what you're saying about the marked tests that makes sense.

https://github.com/ethyca/fidesops/runs/8139612947?check_suite_focus=true

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=== 63 passed, 32 skipped, 1127 deselected, 45 warnings in 67.69s (0:01:07) ====

I think it will be a quick fix.

[sanders41]

It sounds like a bug in the plugin when they are marked something like @pytest.mark.ingegration. From the docs:

Auto mode
When the mode is auto, all discovered async tests are considered asyncio-driven even if they have no @pytest.mark.asyncio marker.

All async fixtures are considered asyncio-driven as well, even if they are decorated with a regular @pytest.fixture decorator instead of dedicated @pytest_asyncio.fixture counterpart.

asyncio-driven means that tests and fixtures are executed by pytest-asyncio plugin.

This mode requires the simplest tests and fixtures configuration and is recommended for default usage unless the same project and its test suite should execute tests from different async frameworks, e.g. asyncio and trio. In this case, auto-handling can break tests designed for other framework; please use strict mode instead.

It's not going to hurt anything for us to mark them like you said so seems like that is the best solution.

[pattisdr]

thanks for narrowing the cause down further, this makes sense