VSCode slow with extension enabled

[azaslavsky]

Type: Bug

I used the Extension bisect feature to figure out what was slowing down my system when working on a Python codebase, and it's the root Python extension, not isort or pylance as I previously suspected. Everything is much slower with the extension enabled: git operations (revert, compare changes), reference lookups, even saving files sometimes takes several seconds.

Extension version: 2023.16.0
VS Code version: Code 1.82.2 (Universal) (abd2f3db4bdb28f9e95536dfa84d8479f1eb312d, 2023-09-14T05:59:47.790Z)
OS version: Darwin arm64 22.6.0
Modes:

System Info

Item	Value
CPUs	Apple M1 Max (10 x 24)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: enabled
Load (avg)	3, 3, 3
Memory (System)	32.00GB (1.35GB free)
Process Argv	--crash-reporter-id 2cf04514-274e-48b1-a654-a4511ede9b53
Screen Reader	yes
VM	0%

[azaslavsky]

I opened dev tools in VSCode, and it had a warning WARN: UNRESPONSIVE extension host: 'ms-python.python' took 71.53390729067382% of 3577.709ms, saved PROFILE here: .... Here is that profile:

ms-python-cpuprofile.txt

Is there any other information I can provide to help debug the problem? For reference, the repository that serves as my open folder is roughly the main branch of https://github.com/getsentry/sentry.

[karrtikr]

Thanks for the bug report! We investigate issues in order based on priority and severity, which includes the impact it has on your ability to use the extension to do productive work, and the number of people affected. If other users come forward and leave a comment demonstrating they are seeing/reproducing the problem then we will raise this issue's priority. Thanks for understanding and the patience!

[azaslavsky]

Is there any guidance you can provide on how I can debug this issue myself? Is there a debug mode in the extension I can enable to figure out where exactly the slowdown is occurring?

[karrtikr]

Here: https://github.com/microsoft/vscode-wiki/blob/main/Performance-Issues.md#profile-the-running-extensions

Logs could also provide some guidance regarding what is being run.

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

XXX

[azaslavsky]

I think I've figured out the solution in my case: I flipped the following two flags to false in my settings.json:

  "python.experiments.enabled": false,
  "python.testing.autoTestDiscoverOnSaveEnabled": false

For reference, the repository I am working on (https://github.com/getsentry/sentry) has ~5400 Python files, and ~23000 tests visible to pytest. I think either the re-indexing of that many tests, or perhaps that feature's interaction with some experimental feature, is what is causing the slowdown.

[karrtikr]

Great, reassigning to testing.

[eleanorjboyd]

Hello! We have tried our best to make our test discovery performant but every save a this point does re-run discovery. Can you try still having the experiment on and just turning off the auto-test discovery and let me know if that helps? Our new testing experiment should be faster than our old architecture so I would love to hear if you are not seeing that on your end. Thanks

[azaslavsky]

Toggling the experiment flag didn't change anything - it is still very slow. I also confirmed that it is related to the test scanning specifically: whenever I manually invoke Test: Refresh Tests from the command palette, my editor becomes very sluggish, and saving a Python file takes >5 seconds. The test refreshing action usually takes 30-60s to complete, so this is quite onerous while it's happening.

[eleanorjboyd]

Thank you for trying it both ways with the experiment. When you run the same command from the command line (are you using pytest or unittest?) how long does it take to return? You can find the command in your logs but try and specify --collect-only for example for pytest to simulate what we run from the extension. Wondering the difference in time between our process and that of pytest/unittest alone.

Secondly, you say the editor gets sluggish, does it freeze or does it just work slower? And this happens after you invoke refresh tests and before you see any changes to the UI?

Thank you for your help!

[azaslavsky]

are you using pytest or unittest?

Pytest only.

When you run the same command from the command line (are you using pytest or unittest?) how long does it take to return?

It varies quite a bit, but the little "clock" appears on the test indicator in the left panel for anywhere from 10 seconds to 2 minutes in the worst case.

You can find the command in your logs but try and specify --collect-only for example for pytest to simulate what we run from the extension. Wondering the difference in time between our process and that of pytest/unittest alone.

Can you give more detailed instructions on how to accomplish this? Where do I pass the --collect-pnly flag? How do I view this data? etc.

[eleanorjboyd]

First in the command palette set your Developer: Set log level to be trace. Thenb in your output panel select Python and you should see a line like: Running pytest discovery with command: -m pytest -p vscode_pytest --collect-only --color=yes -o log_cli=true right after you click refresh. This is the exact command we run so if you just remove our custom plugin (it doesn't work from the command line) and run this command it will run discovery for you in the command line and just exit once it finishes so that's how you can observe the time it takes to do so. So the command you run in the end looks something like python -m pytest --collect-only .....

[eleanorjboyd]

@connor4312, any thoughts on sluggish editor when test discovery is being run for a large workspace? Do you think there is anything on the core side that could help reduce editor lag in this case?

[connor4312]

I think the UI slowness is probably the same as microsoft/vscode#193240 under the hood, so I'll fix that. As a former self-hoster of Sentry, I feel obligated to 😉

It looks like the extension is also loaded fairly heavy as indicated by #22160 (comment). It would probably be worth looking into that on the extension side as well to see if there's anything to optimize.

[connor4312]

I had some time so I made a PR that reduces blocking time in the extension host from 17s -> 950ms when loading the 75k tests from the lsprotocol repo: #22414

[eleanorjboyd]

fyi to those in the thread we will be delaying this PR for about a month as we make other design changes, thanks