Skip to content

ENH make spawn.get_executable return bytes on POSIX - #682

Open
Muhtasim-Munif-Fahim wants to merge 1 commit into
joblib:masterfrom
Muhtasim-Munif-Fahim:get-executable-bytes-posix
Open

Muhtasim-Munif-Fahim wants to merge 1 commit into
joblib:masterfrom
Muhtasim-Munif-Fahim:get-executable-bytes-posix

Conversation

@Muhtasim-Munif-Fahim

Copy link
Copy Markdown

Closes #655

loky.backend.spawn.get_executable now follows multiprocessing.spawn: the interpreter path is stored as bytes on POSIX and str on Windows, through a new set_executable helper (mirroring the stdlib one). That lets us remove the ad-hoc path.encode("utf-8") (and its TODO) from resource_tracker.spawnv_passfds, so the path is now encoded with os.fsencode like in the stdlib rather than forced to UTF-8.

Callers on POSIX (resource_tracker and the vendored stdlib_py314_resource_tracker) pass the value straight to util.spawnv_passfds, which already accepts bytes. The Windows code paths still get a str.

Tests:

  • added test_get_executable_type and test_set_executable_roundtrip in tests/test_loky_backend.py
  • tests/test_loky_backend.py, tests/test_resource_tracker.py, tests/test_reusable_executor.py pass locally (Linux, Python 3.13)
  • black 25.1.0 clean

AI-assistance disclosure: I prepared this change with help from an AI coding assistant. I reviewed it and tested it locally.

Align loky.backend.spawn with multiprocessing.spawn: store the
interpreter path as bytes on POSIX and str on Windows, add a matching
set_executable helper and drop the ad-hoc utf-8 encoding in
resource_tracker.spawnv_passfds.

Closes joblib#655

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

loky.backend.spawn.get_executable should return bytes on POSIX

1 participant