Skip to content

Commit

Permalink
Remove base _CachingProtocolMeta and use beartype's instead
Browse files Browse the repository at this point in the history
Now that beartype/beartype#86 has landed (and made it into a release), we can remove our own base implementation and use that one instead. We still provide our own implementation that allows overriding runtime checking, but it neatly derives from ``beartype``'s.
  • Loading branch information
posita committed Feb 21, 2022
1 parent 514e875 commit c4f184b
Show file tree
Hide file tree
Showing 19 changed files with 61 additions and 232 deletions.
9 changes: 3 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -552,17 +552,14 @@ Alternately, you can download [the source](https://github.com/posita/numerary) a
It has the following runtime dependencies:

* [``typing-extensions``](https://pypi.org/project/typing-extensions/) (with Python <3.9)

``numerary`` will opportunistically use the following, if available at runtime:

* [``beartype``](https://pypi.org/project/beartype/) for yummy runtime type-checking goodness (0.8+)
* [``beartype``](https://pypi.org/project/beartype/) for caching protocols (0.10.1+)
[![Bear-ified™](https://raw.githubusercontent.com/beartype/beartype-assets/main/badge/bear-ified.svg)](https://beartype.rtfd.io/)

If you use ``beartype`` for type-checking your code, but don’t want ``numerary`` to use it internally, set the ``NUMERARY_BEARTYPE`` environment variable to a falsy[^6] value before ``numerary`` is loaded.
``numerary`` will not use ``beartype`` internally unless the ``NUMERARY_BEARTYPE`` environment variable is set to a truthy[^6] value before ``numerary`` is loaded.

[^6]:

I.E., one of: ``0``, ``off``, ``f``, ``false``, and ``no``.
I.E., one of: ``1``, ``on``, ``t``, ``true``, and ``yes``.

See the [hacking quick-start](https://posita.github.io/numerary/0.3/contrib/#hacking-quick-start) for additional development and testing dependencies.

Expand Down
13 changes: 13 additions & 0 deletions docs/notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,19 @@

# ``numerary`` release notes

## [0.4.0](https://github.com/posita/numerary/releases/tag/v0.4.0)

* Now relies on ``#!python beartype.typing.Protocol`` as the underlying caching protocol implementation.
This means that ``beartype`` is now ``numerary``’s only runtime dependency.
(``numerary`` still layers on its own runtime override mechanism via [CachingProtocolMeta][numerary._protocol.CachingProtocolMeta], which derives from ``beartype``’s.)

This decision was not made lightly.
``numerary`` is intended as a temporary work-around.
It’s obsolescence will be something to celebrate.
Caching protocols, however, have much broader performance applications.
They deserve more.
``beartype`` will provide what ``numerary`` was never meant to: a loving, stable, and permanent home.

## [0.3.0](https://github.com/posita/numerary/releases/tag/v0.3.0)

* ~~Removes misleading advice that SCUs offer a performance benefit over merely using caching protocols.
Expand Down
192 changes: 13 additions & 179 deletions numerary/_protocol.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@

from __future__ import annotations

import sys
from abc import abstractmethod
from collections import defaultdict
from typing import TYPE_CHECKING, Any, Dict, Iterable, Set, Tuple, Type, TypeVar, Union
from typing import TYPE_CHECKING, Any, Dict, Set, Tuple, Type, TypeVar

from beartype.typing import Protocol

__all__ = ("CachingProtocolMeta",)

Expand All @@ -22,196 +22,30 @@
_T_co = TypeVar("_T_co", covariant=True)
_TT = TypeVar("_TT", bound="type")

if sys.version_info >= (3, 8):
from typing import _get_protocol_attrs # type: ignore [attr-defined]
from typing import (
Protocol,
SupportsAbs,
SupportsComplex,
SupportsFloat,
SupportsIndex,
SupportsInt,
SupportsRound,
runtime_checkable,
)
else:
from typing_extensions import Protocol, _get_protocol_attrs, runtime_checkable

@runtime_checkable
class SupportsAbs(Protocol[_T_co]):
__slots__: Union[str, Iterable[str]] = ()

@abstractmethod
def __abs__(self) -> _T_co:
pass

@runtime_checkable
class SupportsComplex(Protocol):
__slots__: Union[str, Iterable[str]] = ()

@abstractmethod
def __complex__(self) -> complex:
pass

@runtime_checkable
class SupportsFloat(Protocol):
__slots__: Union[str, Iterable[str]] = ()

@abstractmethod
def __float__(self) -> float:
pass

@runtime_checkable
class SupportsIndex(Protocol):
__slots__: Union[str, Iterable[str]] = ()

@abstractmethod
def __index__(self) -> int:
pass

@runtime_checkable
class SupportsInt(Protocol):
__slots__: Union[str, Iterable[str]] = ()

@abstractmethod
def __int__(self) -> int:
pass

@runtime_checkable
class SupportsRound(Protocol[_T_co]):
__slots__: Union[str, Iterable[str]] = ()

@abstractmethod
def __round__(self, ndigits: int = 0) -> _T_co:
pass


if TYPE_CHECKING:
# Warning: Deep typing voodoo ahead. See
# <https://github.com/python/mypy/issues/11614>.
from abc import ABCMeta as _ProtocolMeta
from abc import ABCMeta as _CachingProtocolMeta
else:
_ProtocolMeta = type(Protocol)


class _CachingProtocolMeta(_ProtocolMeta):
r"""
TODO
"""

_abc_inst_check_cache: Dict[Type, bool]

def __new__(
mcls: Type[_TT],
name: str,
bases: Tuple[Type, ...],
namespace: Dict[str, Any],
**kw: Any,
) -> _TT:
# See <https://github.com/python/mypy/issues/9282>
cls = super().__new__(mcls, name, bases, namespace, **kw) # type: ignore [misc]

# This is required because, despite deriving from typing.Protocol, our
# redefinition below gets its _is_protocol class member set to False. It being
# True is required for compatibility with @runtime_checkable. So we lie to tell
# the truth.
cls._is_protocol = True

# Prefixing this class member with "_abc_" is necessary to prevent it from being
# considered part of the Protocol. (See
# <https://github.com/python/cpython/blob/main/Lib/typing.py>.)
cls._abc_inst_check_cache = {}

return cls

def __instancecheck__(cls, inst: Any) -> bool:
try:
# This has to stay *super* tight! Even adding a mere assertion can add ~50%
# to the best case runtime!
return cls._abc_inst_check_cache[type(inst)]
except KeyError:
# If you're going to do *anything*, do it here. Don't touch the rest of this
# method if you can avoid it.
inst_t = type(inst)
bases_pass_muster = True

for base in cls.__bases__:
# TODO(posita): Checking names seems off to me. Is there a better way?
if base is cls or base.__name__ in ("Protocol", "Generic", "object"):
continue

if not isinstance(inst, base):
bases_pass_muster = False
break

cls._abc_inst_check_cache[
inst_t
] = bases_pass_muster and cls._check_only_my_attrs(inst)

return cls._abc_inst_check_cache[inst_t]

def _check_only_my_attrs(cls, inst: Any) -> bool:
attrs = set(cls.__dict__)
attrs.update(cls.__dict__.get("__annotations__", {}))
attrs.intersection_update(_get_protocol_attrs(cls))

for attr in attrs:
if not hasattr(inst, attr):
return False
elif callable(getattr(cls, attr, None)) and getattr(inst, attr) is None:
return False

return True
_CachingProtocolMeta = type(Protocol)


class CachingProtocolMeta(_CachingProtocolMeta):
# TODO(posita): Add more precise link to beartype.typing.Protocol documentation once
# it becomes available.
r"""
Stand-in for ``#!python typing.Protocol``’s base class that caches results of
``#!python __instancecheck__``, (which is otherwise [really 🤬ing
expensive](https://github.com/python/mypy/issues/3186#issuecomment-885718629)).
(When this was introduced, it resulted in about a 5× performance increase for
[``dyce``](https://github.com/posita/dyce)’s unit tests, which was the only
benchmark I had at the time.) The downside is that this will yield unpredictable
results for objects whose methods don’t stem from any type (e.g., are assembled at
runtime). I don’t know of any real-world case where that would be true. We’ll jump
off that bridge when we come to it.
Note that one can make an existing protocol a caching protocol through inheritance,
but in order to be ``@runtime_checkable``, the parent protocol also has to be
``@runtime_checkable``.
``` python
>>> from abc import abstractmethod
>>> from numerary.types import Protocol, runtime_checkable
>>> @runtime_checkable
... class _MyProtocol(Protocol): # plain vanilla protocol
... @abstractmethod
... def myfunc(self, arg: int) -> str:
... pass
>>> @runtime_checkable # redundant, but useful for documentation
... class MyProtocol(
... _MyProtocol,
... Protocol,
... metaclass=CachingProtocolMeta, # caching version
... ):
... pass
>>> class MyImplementation:
... def myfunc(self, arg: int) -> str:
... return str(arg * -2 + 5)
>>> my_thing: MyProtocol = MyImplementation()
>>> isinstance(my_thing, MyProtocol)
True
```
An augmented version of the [``#!python
beartype.typing.Protocol``](https://github.com/beartype/beartype) caching protocol
that allows for overriding runtime checks.
"""

_abc_inst_check_cache_overridden: Dict[Type, bool]
_abc_inst_check_cache_listeners: Set[CachingProtocolMeta]

# Defined in beartype.typing.Protocol from which we inherit
_abc_inst_check_cache: Dict[type, bool]

def __new__(
mcls: Type[_TT],
name: str,
Expand Down
24 changes: 4 additions & 20 deletions numerary/bt.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,8 @@

from __future__ import annotations

import logging
import os
import sys
import traceback
import warnings
from typing import TypeVar, cast

__all__ = ("beartype",)
Expand Down Expand Up @@ -46,7 +43,7 @@ def __call__(self, __: _T) -> _T:

beartype: _DecoratorT = identity

_NUMERARY_BEARTYPE = os.environ.get("NUMERARY_BEARTYPE", "on")
_NUMERARY_BEARTYPE = os.environ.get("NUMERARY_BEARTYPE", "no")
_truthy = ("on", "t", "true", "yes")
_falsy = ("off", "f", "false", "no")
_use_beartype_if_available: bool
Expand All @@ -64,19 +61,6 @@ def __call__(self, __: _T) -> _T:
)

if _use_beartype_if_available:
try:
import beartype as _beartype
except ImportError:
pass
except Exception:
logging.getLogger(__name__).warning(
"unexpected error when attempting to load beartype"
)
logging.getLogger(__name__).debug(traceback.format_exc())
else:
if _beartype.__version_info__ >= (0, 8):
beartype = cast(_DecoratorT, _beartype.beartype)
else:
warnings.warn(
f"beartype>=0.8 required, but beartype=={_beartype.__version__} found; disabled"
)
import beartype as _beartype

beartype = cast(_DecoratorT, _beartype.beartype)
15 changes: 8 additions & 7 deletions numerary/types.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,15 @@
overload,
)

from beartype.typing import SupportsAbs as _SupportsAbs
from beartype.typing import SupportsComplex as _SupportsComplex
from beartype.typing import SupportsFloat as _SupportsFloat
from beartype.typing import SupportsIndex as _SupportsIndex
from beartype.typing import SupportsInt as _SupportsInt
from beartype.typing import SupportsRound as _SupportsRound
from beartype.typing import runtime_checkable

from ._protocol import CachingProtocolMeta
from ._protocol import SupportsAbs as _SupportsAbs
from ._protocol import SupportsComplex as _SupportsComplex
from ._protocol import SupportsFloat as _SupportsFloat
from ._protocol import SupportsIndex as _SupportsIndex
from ._protocol import SupportsInt as _SupportsInt
from ._protocol import SupportsRound as _SupportsRound
from ._protocol import runtime_checkable
from .bt import beartype

if TYPE_CHECKING:
Expand Down
14 changes: 7 additions & 7 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -46,13 +46,11 @@ packages = numerary
python_requires = >=3.7
install_requires =
typing-extensions>=3.10;python_version<'3.9'
setup_requires =
typing-extensions>=3.10;python_version<'3.9'
beartype>=0.10.1

[options.extras_require] # ------------------------------------------------------------

dev =
beartype>=0.8,!=0.9.0 # for testing
numpy # for testing
pre-commit
setuptools-scm
Expand Down Expand Up @@ -90,7 +88,6 @@ commands =
!debug: pytest --cov=numerary --numprocesses {env:NUMBER_OF_PROCESSORS:auto} {posargs}
deps =
--editable .
beartype: beartype>=0.8,!=0.9.0
!pypy37-!pypy38: numpy
pytest
# Because ${HOME} is not passed, ~/.gitconfig is not read. To overcome this, port any
Expand All @@ -108,6 +105,8 @@ deps =
passenv =
PYTHONBREAKPOINT
setenv =
beartype: NUMERARY_BEARTYPE = yes
!beartype: NUMERARY_BEARTYPE = no
PYTHONWARNINGS = {env:PYTHONWARNINGS:ignore}

[testenv:assets] # --------------------------------------------------------------------
Expand Down Expand Up @@ -149,15 +148,16 @@ whitelist_externals =

commands =
pre-commit run --all-files --show-diff-on-failure
beartype: mypy --config-file={toxinidir}/pyproject.toml --warn-unused-ignores .
!beartype: mypy --config-file={toxinidir}/pyproject.toml .
mypy --config-file={toxinidir}/pyproject.toml --warn-unused-ignores .
{toxinidir}/helpers/mypy-doctests.py -a=--config-file={toxinidir}/pyproject.toml .
deps =
--editable .
beartype: beartype>=0.8,!=0.9.0
mypy>=0.931
pre-commit
sympy
setenv =
beartype: NUMERARY_BEARTYPE = yes
!beartype: NUMERARY_BEARTYPE = no

[flake8] # ----------------------------------------------------------------------------

Expand Down
2 changes: 1 addition & 1 deletion tests/test_integral_like.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@
from typing import cast

import pytest
from beartype import beartype

from numerary import IntegralLike
from numerary.bt import beartype
from numerary.types import __ceil__, __floor__, __trunc__

from .numberwang import (
Expand Down
Loading

0 comments on commit c4f184b

Please sign in to comment.