Paths on Windows example flutter project result in errors

[pushkinman]

Hello @IVLIVS-III, I've successfully launched example flutter project on macOS, but on Windows I am getting the following errors.

In python_ffi_cpython.dart file on line 47 - await tmpZipFile.delete();

Exception has occurred. PathAccessException (PathAccessException: Cannot delete file, path = 'C:\Users\Anton\AppData\Roaming\com.example\example/python_ffi/lib/python3.11.zip' (OS Error: The process cannot access the file because it is being used by another process. , errno = 32))

As you can see from the error log the path is incorrect and has / and \ in different places. Are you planning on fixing this anytime soon?

P.S. After fixing the path in this particular place, I've found out that the issue starts to show up in other places.

[IVLIVS-III]

Hi @pushkinman thanks for this bug report pinpointing the exact issue.
I will try to fix this within the current week.

As I primarily develop and test on MacOS, I sometimes miss cases where I incorrectly use / as path separators on all platforms.

If you come across any other occurrences of the same bug, please comment below. This would speed up fixing the issue.

[maxim-saplin]

@pushkinman , I believe we came across 3 issues when running on WIndows:

1. Wrong slashes on Windows -> as pointed out above
2. FileStream not closed -> error that caused inability to delete file
3. Something wrong with Python runtime paths -> there were odd paths in Debug output and some error on the Python side when hitting breakpoints in main.dart

Could you please elaborate?

[IVLIVS-III]

@maxim-saplin please open separate issues to track your 2nd and 3rd points.
I am committed to provide cross-platform support as best as I can.

Please keep in mind, that this project is part of my bachelors thesis at the moment, thus implementing all research questions on a primary platform (MacOS in my case) is the higher priority.

However, I'll gladly improve Windows support along the way. This requires knowledge of issues specific to Windows including their cause and symptoms.
In order to prevent confusion, I'd like to track each issue separately.

Thank you for your valuable feedback.

[pushkinman]

Yes, first we fixed the slashes from / to \\ in files dart_python_ffi\packages\python_ffi_cpython\lib\python_ffi_cpython.dart and dart_python_ffi\packages\python_ffi_cpython_dart\lib\src\python_ffi_cpython_dart_base.dart.

Secondly, we've closed the file stream in method extractPythonStdLibZip by adding line await inputStream.close(); at the end (this is in file dart_python_ffi\packages\python_ffi_cpython_dart\lib\src\python_ffi_cpython_dart_base.dart).

Finally, the main error was encountered in main.dart on line 12 - await PythonFfi.instance.initialize();

Exception has occurred.
PythonFfiException (PythonFfiException: Python runtime exited with status 0 in function init_fs_encoding: failed to get the Python codec of the filesystem encoding)

[maxim-saplin]

For working with paths in platform agnostic way I'd suggest to go with: https://pub.dev/packages/path
And do smth like this:

import 'package:path/path.dart' as p;
p.join('directory', 'file.txt');

Don't do string concatenation with slashes.

[maxim-saplin]

@IVLIVS-III, rootBundle.load() always expects / slashes, p.join() will break asset loading, the original approach for addressing bundled assets is OK for all platforms:

 final ByteData zipFile = await rootBundle
        .load("packages/python_ffi_cpython/assets/python$_version.zip");

Only file APIs need slashes fixed via path.

[IVLIVS-III]

@IVLIVS-III, rootBundle.load() expects / slashes, p.join() will break asset loading, the original approach for addressing bundled assets is OK for all platforms:

 final ByteData zipFile = await rootBundle
        .load("packages/python_ffi_cpython/assets/python$_version.zip");

Only file APIs need slashes fixed via path.

Thanks for the response, I realized this myself during testing.

[IVLIVS-III]

@maxim-saplin @pushkinman If you have spare time, please test if the current main works on your windows systems as well.

If any of the above issues persists, I'll reopen this issue.

[maxim-saplin]

@maxim-saplin @pushkinman If you have spare time, please test if the current main works on your windows systems as well.

If any of the above issues persists, I'll reopen this issue.

We'll do, thanks for quick turnaround!

[pushkinman]

@IVLIVS-III The example project still does not work after the fix. Here is the log:

Launching lib\main.dart on Windows in debug mode...
lib\main.dart:1
√  Built build\windows\runner\Debug\example.exe.
Connecting to VM Service at ws://127.0.0.1:50354/PgO4aU0b_qk=/ws
Could not find platform independent libraries <prefix>
Python path configuration:
  PYTHONHOME = (not set)
  PYTHONPATH = (not set)
  program name = 'python'
  isolated = 0
  environment = 1
  user site = 1
  safe_path = 0
  import site = 1
  is in build tree = 0
  stdlib dir = 'C:\Users\Anton\Documents\Flutter\dart_python_ffi\packages\python_ffi\example\Lib'
  sys._base_executable = 'C:\\Users\\Anton\\Documents\\Flutter\\dart_python_ffi\\packages\\python_ffi\\example\\build\\windows\\runner\\Debug\\example.exe'
  sys.base_prefix = 'C:\\Users\\Anton\\Documents\\Flutter\\dart_python_ffi\\packages\\python_ffi\\example'
  sys.base_exec_prefix = 'C:\\Users\\Anton\\Documents\\Flutter\\dart_python_ffi\\packages\\python_ffi\\example'
  sys.platlibdir = 'DLLs'
  sys.executable = 'C:\\Users\\Anton\\Documents\\Flutter\\dart_python_ffi\\packages\\python_ffi\\example\\build\\windows\\runner\\Debug\\example.exe'
  sys.prefix = 'C:\\Users\\Anton\\Documents\\Flutter\\dart_python_ffi\\packages\\python_ffi\\example'
  sys.exec_prefix = 'C:\\Users\\Anton\\Documents\\Flutter\\dart_python_ffi\\packages\\python_ffi\\example'
  sys.path = [
    'C:\\Users\\Anton\\Documents\\Flutter\\dart_python_ffi\\packages\\python_ffi\\example\\build\\windows\\runner\\Debug\\python311.zip',
    'C:\\Users\\Anton\\Documents\\Flutter\\dart_python_ffi\\packages\\python_ffi\\example',
    'C:\\Users\\Anton\\Documents\\Flutter\\dart_python_ffi\\packages\\python_ffi\\example\\Lib',
    'C:\\Users\\Anton\\Documents\\Flutter\\dart_python_ffi\\packages\\python_ffi\\example\\build\\windows\\runner\\Debug',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'
Current thread 0x00000bfc (most recent call first):
  <no Python frame>
Lost connection to device.
Exited (sigterm)

[IVLIVS-III]

@pushkinman ahhh right thanks, I've seen this kind of log multiple times before.

The following line points to an incorrect location:

stdlib dir = 'C:\Users\Anton\Documents\Flutter\dart_python_ffi\packages\python_ffi\example\Lib'

I think, it should be 'C:\Users\Anton\Documents\Flutter\dart_python_ffi\packages\python_ffi\example\build\windows\runner\Debug' instead.

I'll look into this tomorrow.

I only tested the examples in packages/examples but not the one in packages/python_ffi/example.

In the meantime… do the other example projects in packages/examples work?

[IVLIVS-III]

Closed in favor of #7

[pushkinman]

@pushkinman ahhh right thanks, I've seen this kind of log multiple times before.

The following line points to an incorrect location:

stdlib dir = 'C:\Users\Anton\Documents\Flutter\dart_python_ffi\packages\python_ffi\example\Lib'

I think, it should be 'C:\Users\Anton\Documents\Flutter\dart_python_ffi\packages\python_ffi\example\build\windows\runner\Debug' instead.

I'll look into this tomorrow.

I only tested the examples in packages/examples but not the one in packages/python_ffi/example.

In the meantime… do the other example projects in packages/examples work?

Let me check

[pushkinman]

@IVLIVS-III I am having trouble launching example projects, could you explain how it should be done?

[IVLIVS-III]

basic_cli_adder: within the packages/examples/basic_cli_adder directory run dart run .\bin\basic_cli_adder.dart -l <libPath> 4 7 where <libPath> is replaced by a path to the Python DLL.

This should be a file called python311.dll somewhere on your system. If not, download and extract the official Python 3.11.3 embeddable package for Windows from https://www.python.org/downloads/release/python-3113/.

This should compute 4 + 7 = 11.

Run dart run .\bin\basic_cli_adder.dart --help for the complete usage.

basic_dataclass:
Run dart run .\bin\basic_dataclass.dart --help for the usage within packages/examples/basic_dataclass

pytimeparse_dart:
The entrypoint is dart run .\example\pytimeparse_dart_example.dart --help

[IVLIVS-III]

lark: only for personal testing, not intended to be runnable.

fj_playground:
Just run flutter run -d windows

[pushkinman]

basic_cli_adder work correctly

PS C:\Users\Anton\Documents\Flutter\dart_python_ffi\packages\examples\basic_cli_adder> dart run .\bin\basic_cli_adder.dart -l C:\Users\Anton\Downloads\python-3.11.3-embed-amd64\python311.dll 4 7
4 + 7 = 11