Test Discovery fails with ModuleNotFoundError while command is working on console

[dovahgab]

Issue Type: Bug

Behaviour

Expected vs. Actual

note: issue similar to #10005
Tests discovery should detect tests, while it does not.

Steps to reproduce:

Project setup, in a basic pipenv environment:

  │   ├── __init__.py
  │   ├── mymodule
  ├── tests
  │   ├── __init__.py
  │   ├── test_mymodule.py

Test discovery on the testing extension fails with
ModuleNotFoundError: No module named 'mymodule'

Manual run with
pipenv run python3 -m pytest tests/test_*.py
works well.

setting.json content:

{
    "python.testing.pytestArgs": [
        "tests"
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.pytestEnabled": true
}

no pytest config file or so.

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.8.10
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Global
• Value of the python.languageServer setting: Default

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

==================================== ERRORS ====================================
________________ ERROR collecting tests/test_inspection_unit.py ________________
ImportError while importing test module 'path/to/file/test_1.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_1.py:2: in <module>
    from mymodule import class1
E   ModuleNotFoundError: No module named 'mymodule'

User Settings

languageServer: "Pylance"

testing
• pytestArgs: "<placeholder>"
• pytestEnabled: true

Extension version: 2022.10.1
VS Code version: Code 1.69.1 (b06ae3b2d2dbfe28bca3134cc6be65935cdfea6a, 2022-07-12T08:28:29.328Z)
OS version: Linux x64 5.15.0-41-generic
Restricted Mode: No

System Info

Item	Value
CPUs	AMD Ryzen 5 3500U with Radeon Vega Mobile Gfx (8 x 1207)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_renderer: enabled_on video_decode: disabled_software video_encode: disabled_software vulkan: disabled_off webgl: enabled webgl2: enabled
Load (avg)	2, 2, 2
Memory (System)	6.72GB (0.57GB free)
Process Argv	--unity-launch --crash-reporter-id 70910446-bfe6-4cdf-a0cf-0efcd70e2fe8
Screen Reader	no
VM	0%
DESKTOP_SESSION	ubuntu
XDG_CURRENT_DESKTOP	Unity
XDG_SESSION_DESKTOP	ubuntu
XDG_SESSION_TYPE	x11

A/B Experiments

vsliv368:30146709
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
pythonvspyl392:30443607
vserr242:30382549
pythontb:30283811
vsjup518:30340749
pythonvspyt551:30345470
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
pythondataviewer:30285071
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593cf:30376535
vsc1dst:30438360
pythonvs932:30410667
wslgetstarted:30449410
vscscmwlcmt:30465135
cppdebug:30492333
pylanb8912cf:30520717
vsclangdf:30486550

[srobertjames]

I'm having a very similar problem. pytest from command line works perfectly. From vscode, however, it fails to discover tests, giving a similar import error.

I believe the root cause may be similar to what's described here: https://medium.com/@dirk.avery/pytest-modulenotfounderror-no-module-named-requests-a770e6926ac5

• pytest imports modules differently depending on how pytest was installed
• If pytest is installed one way, it can't find imports that are in your local code (and not system wide)
• My tests use import of local code from the workspace
• Therefore, some installs (like the one vscode uses) of pytest can't find them
• But other installs (like the one vscode uses) do find them

This all follows from the article linked above ; however, I don't know how to fix it.

This Stackoverflow thread is also relevant: https://stackoverflow.com/questions/10253826/path-issue-with-pytest-importerror-no-module-named-yadayadayada

[karthiknadig]

@dovahgab Can you confirm that the python used to launch the test is the same as the one used by pipenv? from the logs it doesn't look like it is using the same python.

@srobertjames We are working a designing a more robust test runner that works like how you expect pytest to work. Currently we import pytest first and then do discovery, but when running we use pytest directly. This creates mismatch between discovery and running. So, we have plans to make both work the same way.

[srobertjames]

@karthiknadig Thanks! Is there a workaround in the meantime? I'd like to be able to run pytest from vscode so that I can use its interactive debugging features (eg set breakpoints).

[karthiknadig]

You can always create a launch configuration that can launch your tests. But you may have to edit the args to filter tests you want to focus on, but it should get you all the debugging capabilities.

[g1augusto]

I have resolved this by adding a pyproject.toml in the Workspace root folder with the following data:

[tool.pytest.ini_options]
pythonpath = [
  "."
]

give it a try

[srobertjames]

@g1augusto Works perfectly!!! Thank you!

How can I see the test output?

[g1augusto]

You can run the test in debug mode and look at the VScode debug output window

…

On Wed, Jul 27, 2022, 03:14 srobertjames ***@***.***> wrote: @g1augusto <https://github.com/g1augusto> Works perfectly!!! Thank you! How can I see the test output? — Reply to this email directly, view it on GitHub <#19486 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGJVDEA4LLDGWX4IZ5RCPNTVWCEODANCNFSM53VZVUTQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[srobertjames]

@g1augusto To clarify: The toml file allows vscode to run their tests, and show pass or fail, but not any output. When I click the "Show Output" button, the test log console shows:

Run completed, parsing output
Error while running tests: test_myclass
TypeError: Cannot read properties of null (reading 'testsuites')

This suggests to me that the toml file allows vscode to run the tests but not find their output. Do you agree?

[g1augusto]

I can show you how it is on my setup, I can see both output and debug console

Output:

Debug Console:

Terminal (to compare how it is a manual call):

I can tell you I have these parameters set for the pytest call in VScode settings:

[srobertjames]

Thank you. While I'm not able to get all the output, this is still a huge step forward.

[dovahgab]

hi, sorry for lagging, and thanks for your answer
@karthiknadig not sure why it could use a different version, how can I confirm that?
if you are referring to the ModuleNotFoundError and the python output I reported, the latter is what I can see from the tooltip on the python testing extension in vscode, where it fails discovering tests:

I was looking for a solution on this for the same reason, debugging tests - at first I worked it out invoking pytest in a python script, such as

import pytest
retcode = pytest.main(["tests"])

and then debugging it.
@g1augusto thanks! it works for me as well

[multimeric]

I've been hitting this error, which is super annoying. And all the mechanisms to configure this via the settings.json are mostly ignored by vscode. However one thing does work, which is the environment variables file.

Just by running echo 'PYTHONPATH=.' > .env at the root of my project, test discovery suddenly worked again.

The reason for this is that the .env file is automatically sourced by vscode when it does anything python related, and setting the PYTHONPATH like this ensures that your module is imported, since that's kind of like the working directory for the interpreter.

[shaharyuval]

I'm having the exact same problems described above, but with unittest instead of pytest.
Is there a way to tweak the above fixes to make unittest test discovery work?

[eleanorjboyd]

Hello! We have just finished our testing rewrite and are beginning the roll out to users. If you are able, it would be very helpful to know if your issue still exists on the rewrite! To try it yourself, add this setting to your users settings.json "python.experiments.optInto": ["pythonTestAdapter"].

You can confirm you have the rewrite enabled by setting "python.analysis.logLevel": "Trace", in your user settings then check for Experiment 'pythonTestAdapter' is active in your python logs.

Let me know if the rewrite fixes your issue. Thanks!

[dovahgab]

Thanks, the repo I was working on has changed quite a lot.
I tried the test discovery (without changing user settings) and it worked fine - maybe any other having the issue can have a try on it

[eleanorjboyd]

@dovahgab that is great news! I am going to close this to try and reduce our backlog of issues but if it isn't working for anyone still please comment to reopen this issue or create a new one. Thanks!