From 139443bb5a35c423ff0b20e6657e81cb3acda5e8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Sun, 22 Dec 2024 12:36:13 +0100 Subject: [PATCH 1/6] improve changelog --- Doc/library/multiprocessing.rst | 9 +++++++++ Doc/whatsnew/3.14.rst | 20 +++++++++++++++----- 2 files changed, 24 insertions(+), 5 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index 783cb025826483..e68f4703a19947 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -107,6 +107,8 @@ Contexts and start methods Depending on the platform, :mod:`multiprocessing` supports three ways to start a process. These *start methods* are + .. _multiprocessing-start-method-spawn: + *spawn* The parent process starts a fresh Python interpreter process. The child process will only inherit those resources necessary to run @@ -117,6 +119,8 @@ to start a process. These *start methods* are Available on POSIX and Windows platforms. The default on Windows and macOS. + .. _multiprocessing-start-method-fork: + *fork* The parent process uses :func:`os.fork` to fork the Python interpreter. The child process, when it begins, is effectively @@ -137,6 +141,8 @@ to start a process. These *start methods* are raise a :exc:`DeprecationWarning`. Use a different start method. See the :func:`os.fork` documentation for further explanation. + .. _multiprocessing-start-method-forkserver: + *forkserver* When the program starts and selects the *forkserver* start method, a server process is spawned. From then on, whenever a new process @@ -2987,6 +2993,9 @@ Beware of replacing :data:`sys.stdin` with a "file like object" For more information, see :issue:`5155`, :issue:`5313` and :issue:`5331` +_multiprocessing-programming-spawn: +_multiprocessing-programming-forkserver: + The *spawn* and *forkserver* start methods ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/Doc/whatsnew/3.14.rst b/Doc/whatsnew/3.14.rst index 97a37a82f76b9b..d776d8facbbf44 100644 --- a/Doc/whatsnew/3.14.rst +++ b/Doc/whatsnew/3.14.rst @@ -485,17 +485,27 @@ multiprocessing --------------- * The default start method (see :ref:`multiprocessing-start-methods`) changed - from *fork* to *forkserver* on platforms other than macOS & Windows where - it was already *spawn*. If you require the threading incompatible *fork* - start method you must explicitly request it using a context from - :func:`multiprocessing.get_context` (preferred) or change the default via - :func:`multiprocessing.set_start_method`. + from :ref:`fork ` to :ref:`forkserver + ` on platforms other than macOS & + Windows where it was already :ref:`spawn . + + If the threading incompatible *fork* method is required, you must explicitly + request it via a context from :func:`multiprocessing.get_context` (preferred) + or change the default via :func:`multiprocessing.set_start_method`. + + See :ref:`forkserver restrictions ` + for information and differences with the *fork* method and how this change + may affect existing code with mutable global shared variables and/or shared + objects that can not be automatically :mod:`pickled `. + (Contributed by Gregory P. Smith in :gh:`84559`.) + * :mod:`multiprocessing`'s ``"forkserver"`` start method now authenticates its control socket to avoid solely relying on filesystem permissions to restrict what other processes could cause the forkserver to spawn workers and run code. (Contributed by Gregory P. Smith for :gh:`97514`.) + * The :ref:`multiprocessing proxy objects ` for *list* and *dict* types gain previously overlooked missing methods: From 64c636eacc8b0f3312592803688de1de3b93069e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Sun, 22 Dec 2024 12:41:18 +0100 Subject: [PATCH 2/6] fix CI --- Doc/whatsnew/3.14.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/whatsnew/3.14.rst b/Doc/whatsnew/3.14.rst index d776d8facbbf44..ca63b49b18bb55 100644 --- a/Doc/whatsnew/3.14.rst +++ b/Doc/whatsnew/3.14.rst @@ -487,7 +487,7 @@ multiprocessing * The default start method (see :ref:`multiprocessing-start-methods`) changed from :ref:`fork ` to :ref:`forkserver ` on platforms other than macOS & - Windows where it was already :ref:`spawn . + Windows where it was already :ref:`spawn `. If the threading incompatible *fork* method is required, you must explicitly request it via a context from :func:`multiprocessing.get_context` (preferred) From 50e482c33e39fe5b57962cc31b0dd22bad3ab349 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Sun, 22 Dec 2024 12:54:21 +0100 Subject: [PATCH 3/6] fix labels --- Doc/library/multiprocessing.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index e68f4703a19947..1b1c6d74e5197c 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -2993,8 +2993,8 @@ Beware of replacing :data:`sys.stdin` with a "file like object" For more information, see :issue:`5155`, :issue:`5313` and :issue:`5331` -_multiprocessing-programming-spawn: -_multiprocessing-programming-forkserver: +.. _multiprocessing-programming-spawn: +.. _multiprocessing-programming-forkserver: The *spawn* and *forkserver* start methods ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ From 512769806a45183a413c5e48869cd53b2bee6c7a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Sun, 22 Dec 2024 13:24:41 +0100 Subject: [PATCH 4/6] highlight changes --- Doc/whatsnew/3.14.rst | 41 ++++++++++++++++++++++++++++++++++------- 1 file changed, 34 insertions(+), 7 deletions(-) diff --git a/Doc/whatsnew/3.14.rst b/Doc/whatsnew/3.14.rst index ca63b49b18bb55..42e2ea49dffbee 100644 --- a/Doc/whatsnew/3.14.rst +++ b/Doc/whatsnew/3.14.rst @@ -69,6 +69,17 @@ Summary -- release highlights * :ref:`PEP 741: Python Configuration C API ` +Incompatible changes +==================== + +On POSIX, the default :ref:`start method ` for +:mod:`multiprocessing` and :class:`~concurrent.futures.ProcessPoolExecutor` +switches from *fork* to *forkserver*. + +See :ref:`(1) ` and +:ref:`(2) ` for details. + + New features ============ @@ -300,13 +311,27 @@ concurrent.futures same process) to Python code. This is separate from the proposed API in :pep:`734`. (Contributed by Eric Snow in :gh:`124548`.) -* The default ``ProcessPoolExecutor`` start method (see - :ref:`multiprocessing-start-methods`) changed from *fork* to *forkserver* on - platforms other than macOS & Windows. If you require the threading - incompatible *fork* start method you must explicitly request it by - supplying a *mp_context* to :class:`concurrent.futures.ProcessPoolExecutor`. + +.. _whatsnew314-concurrent-futures-start-method: + +* The default :class:`~concurrent.futures.ProcessPoolExecutor` + :ref:`start method ` changed + from :ref:`fork ` to :ref:`forkserver + ` on platforms other than macOS and + Windows where it was already :ref:`spawn `. + + If the threading incompatible *fork* method is required, you must explicitly + request it by supplying a multiprocessing context *mp_context* to + :class:`~concurrent.futures.ProcessPoolExecutor`. + + See :ref:`forkserver restrictions ` + for information and differences with the *fork* method and how this change + may affect existing code with mutable global shared variables and/or shared + objects that can not be automatically :mod:`pickled `. + (Contributed by Gregory P. Smith in :gh:`84559`.) + ctypes ------ @@ -484,9 +509,11 @@ mimetypes multiprocessing --------------- -* The default start method (see :ref:`multiprocessing-start-methods`) changed +.. _whatsnew314-multiprocessing-start-method: + +* The default :ref:`start method ` changed from :ref:`fork ` to :ref:`forkserver - ` on platforms other than macOS & + ` on platforms other than macOS and Windows where it was already :ref:`spawn `. If the threading incompatible *fork* method is required, you must explicitly From 81ca0d161c606da38c60bc34133e81af0607970a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Sun, 22 Dec 2024 20:23:37 +0100 Subject: [PATCH 5/6] improve wording --- Doc/whatsnew/3.14.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/Doc/whatsnew/3.14.rst b/Doc/whatsnew/3.14.rst index 42e2ea49dffbee..c7a32372307d61 100644 --- a/Doc/whatsnew/3.14.rst +++ b/Doc/whatsnew/3.14.rst @@ -72,9 +72,10 @@ Summary -- release highlights Incompatible changes ==================== -On POSIX, the default :ref:`start method ` for -:mod:`multiprocessing` and :class:`~concurrent.futures.ProcessPoolExecutor` -switches from *fork* to *forkserver*. +On platforms other than macOS and Windows, the default :ref:`start +method ` for :mod:`multiprocessing` +and :class:`~concurrent.futures.ProcessPoolExecutor` switches from +*fork* to *forkserver*. See :ref:`(1) ` and :ref:`(2) ` for details. From 25b44e5b08a4ecd4be1fd6086406651540ae7f7d Mon Sep 17 00:00:00 2001 From: "Gregory P. Smith" Date: Tue, 24 Dec 2024 17:00:59 -0800 Subject: [PATCH 6/6] remove extra space Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com> --- Doc/whatsnew/3.14.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/whatsnew/3.14.rst b/Doc/whatsnew/3.14.rst index c7a32372307d61..1f86c48a40ac7b 100644 --- a/Doc/whatsnew/3.14.rst +++ b/Doc/whatsnew/3.14.rst @@ -330,7 +330,7 @@ concurrent.futures may affect existing code with mutable global shared variables and/or shared objects that can not be automatically :mod:`pickled `. - (Contributed by Gregory P. Smith in :gh:`84559`.) + (Contributed by Gregory P. Smith in :gh:`84559`.) ctypes