The process cannot access the file because it is being used by another process

[mebauer]

What happened?

When I attempt to run the extended demo notebook with python .\docs\notebooks\extended_demo.py, I receive this error:

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: C:\\Users\\MARK~1.BAU\\AppData\\Local\\Temp\\tmpc6pe22qx\\my_first_swmm\\bbox_1\\model_1\\model_1.inp

What did you expect

To reproduce the notebook example in the docs.

Minimal example

I followed the steps in the Contributing guide and ran python .\docs\notebooks\extended_demo.py in Windows PowerShell.

Relevant log output

  ERROR 200: one or more errors in input file.
Traceback (most recent call last):
  File "C:\Users\Mark.Bauer\AppData\Local\miniforge3\envs\sa\lib\shutil.py", line 618, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\MARK~1.BAU\\AppData\\Local\\Temp\\tmpc6pe22qx\\my_first_swmm\\bbox_1\\model_1\\model_1.inp'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Mark.Bauer\AppData\Local\miniforge3\envs\sa\lib\tempfile.py", line 852, in onerror
    _os.unlink(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\MARK~1.BAU\\AppData\\Local\\Temp\\tmpc6pe22qx\\my_first_swmm\\bbox_1\\model_1\\model_1.inp'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Mark.Bauer\AppData\Local\miniforge3\envs\sa\lib\weakref.py", line 667, in _exitfunc
    f()
  File "C:\Users\Mark.Bauer\AppData\Local\miniforge3\envs\sa\lib\weakref.py", line 591, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "C:\Users\Mark.Bauer\AppData\Local\miniforge3\envs\sa\lib\tempfile.py", line 868, in _cleanup
    cls._rmtree(name, ignore_errors=ignore_errors)
  File "C:\Users\Mark.Bauer\AppData\Local\miniforge3\envs\sa\lib\tempfile.py", line 864, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\Users\Mark.Bauer\AppData\Local\miniforge3\envs\sa\lib\shutil.py", line 750, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Users\Mark.Bauer\AppData\Local\miniforge3\envs\sa\lib\shutil.py", line 615, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Users\Mark.Bauer\AppData\Local\miniforge3\envs\sa\lib\shutil.py", line 615, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Users\Mark.Bauer\AppData\Local\miniforge3\envs\sa\lib\shutil.py", line 615, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Users\Mark.Bauer\AppData\Local\miniforge3\envs\sa\lib\shutil.py", line 620, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\Users\Mark.Bauer\AppData\Local\miniforge3\envs\sa\lib\tempfile.py", line 855, in onerror
    cls._rmtree(path, ignore_errors=ignore_errors)
  File "C:\Users\Mark.Bauer\AppData\Local\miniforge3\envs\sa\lib\tempfile.py", line 864, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\Users\Mark.Bauer\AppData\Local\miniforge3\envs\sa\lib\shutil.py", line 750, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Users\Mark.Bauer\AppData\Local\miniforge3\envs\sa\lib\shutil.py", line 601, in _rmtree_unsafe
    onerror(os.scandir, path, sys.exc_info())
  File "C:\Users\Mark.Bauer\AppData\Local\miniforge3\envs\sa\lib\shutil.py", line 598, in _rmtree_unsafe
    with os.scandir(path) as scandir_it:
NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\Users\\MARK~1.BAU\\AppData\\Local\\Temp\\tmpc6pe22qx\\my_first_swmm\\bbox_1\\model_1\\model_1.inp'

Anything else we need to know?

I was able to reproduce this error in the command line as well as inside a Jupyter Notebook. I tried installing the package via pip and conda-forge, but ran into bugs downloading data.

Environment

I cloned swmmanywhere from GitHub and am using Python 3.10. OS is Windows.

[barneydobson]

Thanks @mebauer - I can't seem to reproduce, using the following from contributing:

conda create -n sa python=3.10
conda activate sa
git clone https://github.com/ImperialCollegeLondon/SWMManywhere.git
cd SWMManywhere
pip install -e .[dev,doc]
pytest
python .\docs\notebooks\extended_demo.py

Couple questions to help figure out what's going on:

• Are the tests passing?
• What line is the error occurring? This is presumably occurring during the swmmanywhere.run call which initialises the .inp file - but there are multiple calls to this in the demo, so possibly it is on a subsequent call.
• The log looks like something to do with tempfile - does it run if you edit extended_demo.py to remove temp_dir = tempfile.TemporaryDirectory() and replace base_dir=Path(r"/somewhere/on/your/machine/")

[mebauer]

Hi @barneydobson, thanks for being patient with me. I tried on a 1) MacOS, 2) replacing the temporary directory with a dedicated path, and 3) installed the whitebox binaries from the web, and it was successful. However, I tried to reproduce this exact workflow for Windows and received an error of problem with the input file. Although, the input file was populated.

I will provide additional logs once I have time over the holidays, but good to see that it works on MacOS.

Thanks again for your time.

[barneydobson]

Thanks @mebauer glad you've got it working. The unit tests for a release are performed on ubuntu, macos and windows - and it is not being produced on those.

That said - if you can make a test that reproduces the error on a PR I'd be happy to fix it! On that PR you'd have to also include macos-latest here to get the CI to run the test on macos (assuming you can't reproduce it on windows/ubuntu). No worries if you don't have time especially since you now have a workaround - I will label this issue accordingly.