Skip to content

Commit

Permalink
Merge branch 'main' into iOS-configure-reorg
Browse files Browse the repository at this point in the history
  • Loading branch information
erlend-aasland authored Feb 11, 2024
2 parents 00c8965 + 4821f08 commit 17f6028
Show file tree
Hide file tree
Showing 32 changed files with 901 additions and 157 deletions.
11 changes: 8 additions & 3 deletions Doc/c-api/gcsupport.rst
Original file line number Diff line number Diff line change
Expand Up @@ -83,10 +83,15 @@ rules:
.. versionadded:: 3.12
.. c:function:: TYPE* PyObject_GC_Resize(TYPE, PyVarObject *op, Py_ssize_t newsize)
.. c:macro:: PyObject_GC_Resize(TYPE, op, newsize)
Resize an object allocated by :c:macro:`PyObject_NewVar`. Returns the
resized object or ``NULL`` on failure. *op* must not be tracked by the collector yet.
Resize an object allocated by :c:macro:`PyObject_NewVar`.
Returns the resized object of type ``TYPE*`` (refers to any C type)
or ``NULL`` on failure.
*op* must be of type :c:expr:`PyVarObject *`
and must not be tracked by the collector yet.
*newsize* must be of type :c:type:`Py_ssize_t`.
.. c:function:: void PyObject_GC_Track(PyObject *op)
Expand Down
7 changes: 7 additions & 0 deletions Doc/library/array.rst
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,13 @@ The module defines the following type:
Remove the first occurrence of *x* from the array.


.. method:: clear()

Remove all elements from the array.

.. versionadded:: 3.13


.. method:: reverse()

Reverse the order of the items in the array.
Expand Down
38 changes: 38 additions & 0 deletions Doc/library/queue.rst
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,14 @@ The :mod:`queue` module defines the following classes and exceptions:
on a :class:`Queue` object which is full.


.. exception:: ShutDown

Exception raised when :meth:`~Queue.put` or :meth:`~Queue.get` is called on
a :class:`Queue` object which has been shut down.

.. versionadded:: 3.13


.. _queueobjects:

Queue Objects
Expand Down Expand Up @@ -135,6 +143,8 @@ provide the public methods described below.
immediately available, else raise the :exc:`Full` exception (*timeout* is
ignored in that case).

Raises :exc:`ShutDown` if the queue has been shut down.


.. method:: Queue.put_nowait(item)

Expand All @@ -155,6 +165,9 @@ provide the public methods described below.
an uninterruptible wait on an underlying lock. This means that no exceptions
can occur, and in particular a SIGINT will not trigger a :exc:`KeyboardInterrupt`.

Raises :exc:`ShutDown` if the queue has been shut down and is empty, or if
the queue has been shut down immediately.


.. method:: Queue.get_nowait()

Expand All @@ -177,6 +190,8 @@ fully processed by daemon consumer threads.
Raises a :exc:`ValueError` if called more times than there were items placed in
the queue.

Raises :exc:`ShutDown` if the queue has been shut down immediately.


.. method:: Queue.join()

Expand All @@ -187,6 +202,8 @@ fully processed by daemon consumer threads.
indicate that the item was retrieved and all work on it is complete. When the
count of unfinished tasks drops to zero, :meth:`join` unblocks.

Raises :exc:`ShutDown` if the queue has been shut down immediately.


Example of how to wait for enqueued tasks to be completed::

Expand Down Expand Up @@ -214,6 +231,27 @@ Example of how to wait for enqueued tasks to be completed::
print('All work completed')


Terminating queues
^^^^^^^^^^^^^^^^^^

:class:`Queue` objects can be made to prevent further interaction by shutting
them down.

.. method:: Queue.shutdown(immediate=False)

Shut down the queue, making :meth:`~Queue.get` and :meth:`~Queue.put` raise
:exc:`ShutDown`.

By default, :meth:`~Queue.get` on a shut down queue will only raise once the
queue is empty. Set *immediate* to true to make :meth:`~Queue.get` raise
immediately instead.

All blocked callers of :meth:`~Queue.put` will be unblocked. If *immediate*
is true, also unblock callers of :meth:`~Queue.get` and :meth:`~Queue.join`.

.. versionadded:: 3.13


SimpleQueue Objects
-------------------

Expand Down
6 changes: 3 additions & 3 deletions Doc/reference/datamodel.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1988,8 +1988,8 @@ access (use of, assignment to, or deletion of ``x.name``) for class instances.

.. method:: object.__dir__(self)

Called when :func:`dir` is called on the object. A sequence must be
returned. :func:`dir` converts the returned sequence to a list and sorts it.
Called when :func:`dir` is called on the object. An iterable must be
returned. :func:`dir` converts the returned iterable to a list and sorts it.


Customizing module attribute access
Expand All @@ -2009,7 +2009,7 @@ not found on a module object through the normal lookup, i.e.
the module ``__dict__`` before raising an :exc:`AttributeError`. If found,
it is called with the attribute name and the result is returned.

The ``__dir__`` function should accept no arguments, and return a sequence of
The ``__dir__`` function should accept no arguments, and return an iterable of
strings that represents the names accessible on module. If present, this
function overrides the standard :func:`dir` search on a module.

Expand Down
1 change: 0 additions & 1 deletion Doc/tools/.nitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
Doc/c-api/descriptor.rst
Doc/c-api/exceptions.rst
Doc/c-api/float.rst
Doc/c-api/gcsupport.rst
Doc/c-api/init.rst
Doc/c-api/init_config.rst
Doc/c-api/intro.rst
Expand Down
10 changes: 10 additions & 0 deletions Doc/whatsnew/3.13.rst
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,9 @@ array
It can be used instead of ``'u'`` type code, which is deprecated.
(Contributed by Inada Naoki in :gh:`80480`.)

* Add ``clear()`` method in order to implement ``MutableSequence``.
(Contributed by Mike Zimin in :gh:`114894`.)

ast
---

Expand Down Expand Up @@ -403,6 +406,13 @@ pdb
command line option or :envvar:`PYTHONSAFEPATH` environment variable).
(Contributed by Tian Gao and Christian Walther in :gh:`111762`.)

queue
-----

* Add :meth:`queue.Queue.shutdown` (along with :exc:`queue.ShutDown`) for queue
termination.
(Contributed by Laurie Opperman and Yves Duprat in :gh:`104750`.)

re
--
* Rename :exc:`!re.error` to :exc:`re.PatternError` for improved clarity.
Expand Down
13 changes: 7 additions & 6 deletions Lib/fractions.py
Original file line number Diff line number Diff line change
Expand Up @@ -579,7 +579,8 @@ def __format__(self, format_spec, /):
f"for object of type {type(self).__name__!r}"
)

def _operator_fallbacks(monomorphic_operator, fallback_operator):
def _operator_fallbacks(monomorphic_operator, fallback_operator,
handle_complex=True):
"""Generates forward and reverse operators given a purely-rational
operator and a function from the operator module.
Expand Down Expand Up @@ -666,7 +667,7 @@ def forward(a, b):
return monomorphic_operator(a, Fraction(b))
elif isinstance(b, float):
return fallback_operator(float(a), b)
elif isinstance(b, complex):
elif handle_complex and isinstance(b, complex):
return fallback_operator(complex(a), b)
else:
return NotImplemented
Expand All @@ -679,7 +680,7 @@ def reverse(b, a):
return monomorphic_operator(Fraction(a), b)
elif isinstance(a, numbers.Real):
return fallback_operator(float(a), float(b))
elif isinstance(a, numbers.Complex):
elif handle_complex and isinstance(a, numbers.Complex):
return fallback_operator(complex(a), complex(b))
else:
return NotImplemented
Expand Down Expand Up @@ -830,22 +831,22 @@ def _floordiv(a, b):
"""a // b"""
return (a.numerator * b.denominator) // (a.denominator * b.numerator)

__floordiv__, __rfloordiv__ = _operator_fallbacks(_floordiv, operator.floordiv)
__floordiv__, __rfloordiv__ = _operator_fallbacks(_floordiv, operator.floordiv, False)

def _divmod(a, b):
"""(a // b, a % b)"""
da, db = a.denominator, b.denominator
div, n_mod = divmod(a.numerator * db, da * b.numerator)
return div, Fraction(n_mod, da * db)

__divmod__, __rdivmod__ = _operator_fallbacks(_divmod, divmod)
__divmod__, __rdivmod__ = _operator_fallbacks(_divmod, divmod, False)

def _mod(a, b):
"""a % b"""
da, db = a.denominator, b.denominator
return Fraction((a.numerator * db) % (b.numerator * da), da * db)

__mod__, __rmod__ = _operator_fallbacks(_mod, operator.mod)
__mod__, __rmod__ = _operator_fallbacks(_mod, operator.mod, False)

def __pow__(a, b):
"""a ** b
Expand Down
8 changes: 6 additions & 2 deletions Lib/pathlib/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -587,9 +587,13 @@ def iterdir(self):
def _scandir(self):
return os.scandir(self)

def _make_child_entry(self, entry):
def _direntry_str(self, entry):
# Transform an entry yielded from _scandir() into a path string.
return entry.name if str(self) == '.' else entry.path

def _make_child_direntry(self, entry):
# Transform an entry yielded from _scandir() into a path object.
path_str = entry.name if str(self) == '.' else entry.path
path_str = self._direntry_str(entry)
path = self.with_segments(path_str)
path._str = path_str
path._drv = self.drive
Expand Down
82 changes: 56 additions & 26 deletions Lib/pathlib/_abc.py
Original file line number Diff line number Diff line change
Expand Up @@ -86,19 +86,29 @@ def _select_children(parent_paths, dir_only, follow_symlinks, match):
continue
except OSError:
continue
if match(entry.name):
yield parent_path._make_child_entry(entry)
# Avoid cost of making a path object for non-matching paths by
# matching against the os.DirEntry.name string.
if match is None or match(entry.name):
yield parent_path._make_child_direntry(entry)


def _select_recursive(parent_paths, dir_only, follow_symlinks):
"""Yield given paths and all their subdirectories, recursively."""
def _select_recursive(parent_paths, dir_only, follow_symlinks, match):
"""Yield given paths and all their children, recursively, filtering by
string and type.
"""
if follow_symlinks is None:
follow_symlinks = False
for parent_path in parent_paths:
if match is not None:
# If we're filtering paths through a regex, record the length of
# the parent path. We'll pass it to match(path, pos=...) later.
parent_len = len(str(parent_path._make_child_relpath('_'))) - 1
paths = [parent_path._make_child_relpath('')]
while paths:
path = paths.pop()
yield path
if match is None or match(str(path), parent_len):
# Yield *directory* path that matches pattern (if any).
yield path
try:
# We must close the scandir() object before proceeding to
# avoid exhausting file descriptors when globbing deep trees.
Expand All @@ -108,14 +118,22 @@ def _select_recursive(parent_paths, dir_only, follow_symlinks):
pass
else:
for entry in entries:
# Handle directory entry.
try:
if entry.is_dir(follow_symlinks=follow_symlinks):
paths.append(path._make_child_entry(entry))
# Recurse into this directory.
paths.append(path._make_child_direntry(entry))
continue
except OSError:
pass

# Handle file entry.
if not dir_only:
yield path._make_child_entry(entry)
# Avoid cost of making a path object for non-matching
# files by matching against the os.DirEntry object.
if match is None or match(path._direntry_str(entry), parent_len):
# Yield *file* path that matches pattern (if any).
yield path._make_child_direntry(entry)


def _select_unique(paths):
Expand Down Expand Up @@ -750,8 +768,14 @@ def _scandir(self):
from contextlib import nullcontext
return nullcontext(self.iterdir())

def _make_child_entry(self, entry):
def _direntry_str(self, entry):
# Transform an entry yielded from _scandir() into a path string.
# PathBase._scandir() yields PathBase objects, so use str().
return str(entry)

def _make_child_direntry(self, entry):
# Transform an entry yielded from _scandir() into a path object.
# PathBase._scandir() yields PathBase objects, so this is a no-op.
return entry

def _make_child_relpath(self, name):
Expand All @@ -769,43 +793,49 @@ def glob(self, pattern, *, case_sensitive=None, follow_symlinks=None):

stack = pattern._pattern_stack
specials = ('', '.', '..')
filter_paths = False
deduplicate_paths = False
sep = self.pathmod.sep
paths = iter([self] if self.is_dir() else [])
while stack:
part = stack.pop()
if part in specials:
# Join special component (e.g. '..') onto paths.
paths = _select_special(paths, part)

elif part == '**':
# Consume adjacent '**' components.
# Consume following '**' components, which have no effect.
while stack and stack[-1] == '**':
stack.pop()

# Consume adjacent non-special components and enable post-walk
# regex filtering, provided we're treating symlinks consistently.
# Consume following non-special components, provided we're
# treating symlinks consistently. Each component is joined
# onto 'part', which is used to generate an re.Pattern object.
if follow_symlinks is not None:
while stack and stack[-1] not in specials:
filter_paths = True
stack.pop()
part += sep + stack.pop()

dir_only = bool(stack)
paths = _select_recursive(paths, dir_only, follow_symlinks)
# If the previous loop consumed pattern components, compile an
# re.Pattern object based on those components.
match = _compile_pattern(part, sep, case_sensitive) if part != '**' else None

# Recursively walk directories, filtering by type and regex.
paths = _select_recursive(paths, bool(stack), follow_symlinks, match)

# De-duplicate if we've already seen a '**' component.
if deduplicate_paths:
# De-duplicate if we've already seen a '**' component.
paths = _select_unique(paths)
deduplicate_paths = True

elif '**' in part:
raise ValueError("Invalid pattern: '**' can only be an entire path component")

else:
dir_only = bool(stack)
match = _compile_pattern(part, sep, case_sensitive)
paths = _select_children(paths, dir_only, follow_symlinks, match)
if filter_paths:
# Filter out paths that don't match pattern.
prefix_len = len(str(self._make_child_relpath('_'))) - 1
match = _compile_pattern(pattern._pattern_str, sep, case_sensitive)
paths = (path for path in paths if match(path._pattern_str, prefix_len))
# If the pattern component isn't '*', compile an re.Pattern
# object based on the component.
match = _compile_pattern(part, sep, case_sensitive) if part != '*' else None

# Iterate over directories' children filtering by type and regex.
paths = _select_children(paths, bool(stack), follow_symlinks, match)
return paths

def rglob(self, pattern, *, case_sensitive=None, follow_symlinks=None):
Expand Down Expand Up @@ -854,7 +884,7 @@ def walk(self, top_down=True, on_error=None, follow_symlinks=False):

if is_dir:
if not top_down:
paths.append(path._make_child_entry(entry))
paths.append(path._make_child_direntry(entry))
dirnames.append(entry.name)
else:
filenames.append(entry.name)
Expand Down
Loading

0 comments on commit 17f6028

Please sign in to comment.