Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-128555: Add 'context' keyword parameter to Thread. #128209

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion Doc/library/threading.rst
Original file line number Diff line number Diff line change
Expand Up @@ -334,7 +334,7 @@ since it is impossible to detect the termination of alien threads.


.. class:: Thread(group=None, target=None, name=None, args=(), kwargs={}, *, \
daemon=None)
daemon=None, context=None)

This constructor should always be called with keyword arguments. Arguments
are:
Expand All @@ -359,6 +359,11 @@ since it is impossible to detect the termination of alien threads.
If ``None`` (the default), the daemonic property is inherited from the
current thread.

*context* is the `contextvars.Context` value to use while running the thread.
The default is to inherit the context of the caller of :meth:`~Thread.start`.
nascheme marked this conversation as resolved.
Show resolved Hide resolved
nascheme marked this conversation as resolved.
Show resolved Hide resolved
To start with an empty context, pass a new instance of
:class:`contextvars.Context()`

If the subclass overrides the constructor, it must make sure to invoke the
base class constructor (``Thread.__init__()``) before doing anything else to
the thread.
Expand All @@ -369,6 +374,10 @@ since it is impossible to detect the termination of alien threads.
.. versionchanged:: 3.10
Use the *target* name if *name* argument is omitted.

.. versionchanged:: 3.14
Added the *context* parameter. Previously threads always ran with an empty
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd make this more direct about the behavior change, perhaps something like:

"Threads now inherit the context of the caller of Thread.start instead of starting with an empty context. The context parameter was added. Pass a new contextvars.Context() if your thread requires an empty context."

This default change is a behavior change. Expect that to trip someones existing code up. I don't have the context (pun intended) as to how disruptive that could be. Per the PEP-387 breaking change policy we'd want to wait at least two releases. Which isn't so satisfying given the reasons you want this.

But a compromise could be considered (unsure if this is really wise) if needed: Change the default sooner than the deprecation when running in an experimental free-threading cpython build?

context.

.. method:: start()

Start the thread's activity.
Expand Down
43 changes: 43 additions & 0 deletions Lib/test/test_context.py
Original file line number Diff line number Diff line change
Expand Up @@ -383,6 +383,49 @@ def sub(num):
tp.shutdown()
self.assertEqual(results, list(range(10)))

@isolated_context
@threading_helper.requires_working_threading()
def test_context_thread_inherit(self):
import threading

cvar = contextvars.ContextVar('cvar')

# By default, the context of the caller is inheritied
def run_inherit():
self.assertEqual(cvar.get(), 1)

cvar.set(1)
thread = threading.Thread(target=run_inherit)
thread.start()
thread.join()

# If context=None is passed, the thread has an empty context
def run_empty():
with self.assertRaises(LookupError):
cvar.get()

thread = threading.Thread(target=run_empty, context=None)
thread.start()
thread.join()

# An explicit Context value can also be passed
custom_ctx = contextvars.Context()
custom_var = None

def setup_context():
nonlocal custom_var
custom_var = contextvars.ContextVar('custom')
custom_var.set(2)

custom_ctx.run(setup_context)

def run_custom():
self.assertEqual(custom_var.get(), 2)

thread = threading.Thread(target=run_custom, context=custom_ctx)
thread.start()
thread.join()


# HAMT Tests

Expand Down
7 changes: 5 additions & 2 deletions Lib/test/test_decimal.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@
import random
import inspect
import threading
import contextvars


if sys.platform == 'darwin':
Expand Down Expand Up @@ -1725,8 +1726,10 @@ def test_threading(self):
self.finish1 = threading.Event()
self.finish2 = threading.Event()

th1 = threading.Thread(target=thfunc1, args=(self,))
th2 = threading.Thread(target=thfunc2, args=(self,))
th1 = threading.Thread(target=thfunc1, args=(self,),
context=contextvars.Context())
th2 = threading.Thread(target=thfunc2, args=(self,),
context=contextvars.Context())

th1.start()
th2.start()
Expand Down
25 changes: 22 additions & 3 deletions Lib/threading.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
import sys as _sys
import _thread
import warnings
import contextvars as _contextvars


from time import monotonic as _time
from _weakrefset import WeakSet
Expand Down Expand Up @@ -871,7 +873,7 @@ class Thread:
_initialized = False

def __init__(self, group=None, target=None, name=None,
args=(), kwargs=None, *, daemon=None):
args=(), kwargs=None, *, daemon=None, context=None):
"""This constructor should always be called with keyword arguments. Arguments are:

*group* should be None; reserved for future extension when a ThreadGroup
Expand All @@ -888,6 +890,10 @@ class is implemented.
*kwargs* is a dictionary of keyword arguments for the target
invocation. Defaults to {}.

*context* is the contextvars.Context value to use for the thread. The default
is to inherit the context of the caller. To start with an empty context,
nascheme marked this conversation as resolved.
Show resolved Hide resolved
pass a new instance of contextvars.Context().

If a subclass overrides the constructor, it must make sure to invoke
the base class constructor (Thread.__init__()) before doing anything
else to the thread.
Expand Down Expand Up @@ -917,6 +923,7 @@ class is implemented.
self._daemonic = daemon
else:
self._daemonic = current_thread().daemon
self._context = context
self._ident = None
if _HAVE_THREAD_NATIVE_ID:
self._native_id = None
Expand Down Expand Up @@ -972,9 +979,15 @@ def start(self):

with _active_limbo_lock:
_limbo[self] = self

if self._context is None:
# No context provided, inherit the context of the caller.
self._context = _contextvars.copy_context()

try:
# Start joinable thread
_start_joinable_thread(self._bootstrap, handle=self._handle,
_start_joinable_thread(self._bootstrap,
handle=self._handle,
daemon=self.daemon)
except Exception:
with _active_limbo_lock:
Expand Down Expand Up @@ -1051,7 +1064,13 @@ def _bootstrap_inner(self):
_sys.setprofile(_profile_hook)

try:
self.run()
if self._context is None:
# Run with empty context, matching behaviour of
# threading.local and older versions of Python.
self.run()
else:
# Run with the provided or the inherited context.
self._context.run(self.run)
except:
self._invoke_excepthook(self)
finally:
Expand Down
1 change: 1 addition & 0 deletions Makefile.pre.in
Original file line number Diff line number Diff line change
Expand Up @@ -420,6 +420,7 @@ PARSER_HEADERS= \
# Python

PYTHON_OBJS= \
Python/_contextvars.o \
Python/_warnings.o \
Python/Python-ast.o \
Python/Python-tokenize.o \
Expand Down
1 change: 0 additions & 1 deletion Modules/Setup
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,6 @@ PYTHONPATH=$(COREPYTHONPATH)

#_asyncio _asynciomodule.c
#_bisect _bisectmodule.c
#_contextvars _contextvarsmodule.c
#_csv _csv.c
#_datetime _datetimemodule.c
#_decimal _decimal/_decimal.c
Expand Down
1 change: 0 additions & 1 deletion Modules/Setup.stdlib.in
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@
@MODULE_ARRAY_TRUE@array arraymodule.c
@MODULE__ASYNCIO_TRUE@_asyncio _asynciomodule.c
@MODULE__BISECT_TRUE@_bisect _bisectmodule.c
@MODULE__CONTEXTVARS_TRUE@_contextvars _contextvarsmodule.c
@MODULE__CSV_TRUE@_csv _csv.c
@MODULE__HEAPQ_TRUE@_heapq _heapqmodule.c
@MODULE__JSON_TRUE@_json _json.c
Expand Down
4 changes: 4 additions & 0 deletions Modules/config.c.in
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ extern PyObject* PyInit__imp(void);
extern PyObject* PyInit_gc(void);
extern PyObject* PyInit__ast(void);
extern PyObject* PyInit__tokenize(void);
extern PyObject* PyInit__contextvars(void);
extern PyObject* _PyWarnings_Init(void);
extern PyObject* PyInit__string(void);

Expand All @@ -55,6 +56,9 @@ struct _inittab _PyImport_Inittab[] = {
/* This lives in gcmodule.c */
{"gc", PyInit_gc},

/* This lives in Python/_contextvars.c */
{"_contextvars", PyInit__contextvars},

/* This lives in _warnings.c */
{"_warnings", _PyWarnings_Init},

Expand Down
2 changes: 1 addition & 1 deletion PCbuild/pythoncore.vcxproj
Original file line number Diff line number Diff line change
Expand Up @@ -423,7 +423,6 @@
</ClCompile>
<ClCompile Include="..\Modules\_codecsmodule.c" />
<ClCompile Include="..\Modules\_collectionsmodule.c" />
<ClCompile Include="..\Modules\_contextvarsmodule.c" />
<ClCompile Include="..\Modules\_csv.c" />
<ClCompile Include="..\Modules\_functoolsmodule.c" />
<ClCompile Include="..\Modules\_hacl\Hacl_Hash_MD5.c" />
Expand Down Expand Up @@ -570,6 +569,7 @@
<ClCompile Include="..\PC\config.c" />
<ClCompile Include="..\PC\msvcrtmodule.c" />
<ClCompile Include="..\Python\pyhash.c" />
<ClCompile Include="..\Python\_contextvars.c" />
<ClCompile Include="..\Python\_warnings.c" />
<ClCompile Include="..\Python\asdl.c" />
<ClCompile Include="..\Python\assemble.c" />
Expand Down
6 changes: 3 additions & 3 deletions PCbuild/pythoncore.vcxproj.filters
Original file line number Diff line number Diff line change
Expand Up @@ -1262,6 +1262,9 @@
<ClCompile Include="..\PC\msvcrtmodule.c">
<Filter>PC</Filter>
</ClCompile>
<ClCompile Include="..\Python\_contextvars.c">
<Filter>Python</Filter>
</ClCompile>
<ClCompile Include="..\Python\_warnings.c">
<Filter>Python</Filter>
</ClCompile>
Expand Down Expand Up @@ -1526,9 +1529,6 @@
<ClCompile Include="..\Objects\odictobject.c">
<Filter>Objects</Filter>
</ClCompile>
<ClCompile Include="..\Modules\_contextvarsmodule.c">
<Filter>Modules</Filter>
</ClCompile>
<ClCompile Include="$(zlibDir)\adler32.c">
<Filter>Modules\zlib</Filter>
</ClCompile>
Expand Down
2 changes: 1 addition & 1 deletion Modules/_contextvarsmodule.c → Python/_contextvars.c
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#include "Python.h"

#include "clinic/_contextvarsmodule.c.h"
#include "clinic/_contextvars.c.h"

/*[clinic input]
module _contextvars
Expand Down
1 change: 0 additions & 1 deletion configure.ac
Original file line number Diff line number Diff line change
Expand Up @@ -7709,7 +7709,6 @@ dnl always enabled extension modules
PY_STDLIB_MOD_SIMPLE([array])
PY_STDLIB_MOD_SIMPLE([_asyncio])
PY_STDLIB_MOD_SIMPLE([_bisect])
PY_STDLIB_MOD_SIMPLE([_contextvars])
PY_STDLIB_MOD_SIMPLE([_csv])
PY_STDLIB_MOD_SIMPLE([_heapq])
PY_STDLIB_MOD_SIMPLE([_json])
Expand Down
Loading