ContextQMD
Back to Ruff

Binary operations on instances

crates/ty_python_semantic/resources/mdtest/binary/instances.md

0.15.2115.9 KB
Original Source

Binary operations on instances

Binary operations in Python are implemented by means of magic double-underscore methods.

For references, see:

Operations

We support inference for all Python's binary operators: +, -, *, @, /, //, %, **, <<, >>, &, ^, and |.

py
class A:
    def __add__(self, other) -> "A":
        return self

    def __sub__(self, other) -> "A":
        return self

    def __mul__(self, other) -> "A":
        return self

    def __matmul__(self, other) -> "A":
        return self

    def __truediv__(self, other) -> "A":
        return self

    def __floordiv__(self, other) -> "A":
        return self

    def __mod__(self, other) -> "A":
        return self

    def __pow__(self, other) -> "A":
        return self

    def __lshift__(self, other) -> "A":
        return self

    def __rshift__(self, other) -> "A":
        return self

    def __and__(self, other) -> "A":
        return self

    def __xor__(self, other) -> "A":
        return self

    def __or__(self, other) -> "A":
        return self

class B: ...

reveal_type(A() + B())  # revealed: A
reveal_type(A() - B())  # revealed: A
reveal_type(A() * B())  # revealed: A
reveal_type(A() @ B())  # revealed: A
reveal_type(A() / B())  # revealed: A
reveal_type(A() // B())  # revealed: A
reveal_type(A() % B())  # revealed: A
reveal_type(A() ** B())  # revealed: A
reveal_type(A() << B())  # revealed: A
reveal_type(A() >> B())  # revealed: A
reveal_type(A() & B())  # revealed: A
reveal_type(A() ^ B())  # revealed: A
reveal_type(A() | B())  # revealed: A

Recursive dunder return annotation

py
from __future__ import annotations

class A:
    def __add__(self, other: object) -> type(x + x):  # error: [invalid-type-form]
        ...

x = A()
reveal_type(x + x)  # revealed: Unknown

Reflected

We also support inference for reflected operations:

py
class A:
    def __radd__(self, other) -> "A":
        return self

    def __rsub__(self, other) -> "A":
        return self

    def __rmul__(self, other) -> "A":
        return self

    def __rmatmul__(self, other) -> "A":
        return self

    def __rtruediv__(self, other) -> "A":
        return self

    def __rfloordiv__(self, other) -> "A":
        return self

    def __rmod__(self, other) -> "A":
        return self

    def __rpow__(self, other) -> "A":
        return self

    def __rlshift__(self, other) -> "A":
        return self

    def __rrshift__(self, other) -> "A":
        return self

    def __rand__(self, other) -> "A":
        return self

    def __rxor__(self, other) -> "A":
        return self

    def __ror__(self, other) -> "A":
        return self

class B: ...

reveal_type(B() + A())  # revealed: A
reveal_type(B() - A())  # revealed: A
reveal_type(B() * A())  # revealed: A
reveal_type(B() @ A())  # revealed: A
reveal_type(B() / A())  # revealed: A
reveal_type(B() // A())  # revealed: A
reveal_type(B() % A())  # revealed: A
reveal_type(B() ** A())  # revealed: A
reveal_type(B() << A())  # revealed: A
reveal_type(B() >> A())  # revealed: A
reveal_type(B() & A())  # revealed: A
reveal_type(B() ^ A())  # revealed: A
reveal_type(B() | A())  # revealed: A

Returning a different type

The magic methods aren't required to return the type of self:

py
class A:
    def __add__(self, other) -> int:
        return 1

    def __rsub__(self, other) -> int:
        return 1

class B: ...

reveal_type(A() + B())  # revealed: int
reveal_type(B() - A())  # revealed: int

Non-reflected precedence in general

In general, if the left-hand side defines __add__ and the right-hand side defines __radd__ and the right-hand side is not a subtype of the left-hand side, lhs.__add__ will take precedence:

py
class A:
    def __add__(self, other: "B") -> int:
        return 42

class B:
    def __radd__(self, other: "A") -> str:
        return "foo"

reveal_type(A() + B())  # revealed:  int

# Edge case: C is a subtype of C, *but* if the two sides are of *equal* types,
# the lhs *still* takes precedence
class C:
    def __add__(self, other: "C") -> int:
        return 42

    def __radd__(self, other: "C") -> str:
        return "foo"

reveal_type(C() + C())  # revealed: int

Reflected precedence for subtypes (in some cases)

If the right-hand operand is a subtype of the left-hand operand and has a different implementation of the reflected method, its reflected method may take precedence. The static types do not tell us whether the left operand has runtime class A or a subclass such as B, so both methods can be called at runtime.

py
from typing import Literal

class A:
    def __add__(self, other) -> Literal["left"]:
        return "left"

class B(A):
    def __radd__(self, other) -> Literal["right"]:
        return "right"

reveal_type(A() + B())  # revealed: Literal["right", "left"]

# N.B. Still a subtype of `A`, even though `A` does not appear directly in the class's `__bases__`
class C(B): ...

reveal_type(A() + C())  # revealed: Literal["right", "left"]

Reflected precedence uses runtime classes

IntFlag values are commonly accumulated into a mask that starts at the integer zero. At runtime, the first |= produces a Permission, because reflected-method precedence depends on the operands' runtime classes. The enum literal is not a subtype of the specific integer literal 0, but its runtime class is a strict subclass of int. The same applies when the right operand is a TypeVar whose upper bound has that runtime-class relationship:

py
from enum import IntFlag, auto
from typing import TypeVar

class Permission(IntFlag):
    READ = auto()
    WRITE = auto()

def permissions_for(editable: bool) -> Permission:
    permissions = 0
    permissions |= Permission.READ
    reveal_type(permissions)  # revealed: Literal[Permission.READ]

    if editable:
        permissions |= Permission.WRITE

    return permissions

P = TypeVar("P", bound=Permission)

def add_permission(permission: P) -> P:
    reveal_type(0 | permission)  # revealed: P@add_permission
    return 0 | permission

Runtime-class precedence ignores generic specializations

Generic specializations do not exist at runtime, so they cannot affect whether the right operand's runtime class is a strict subclass of the left operand's runtime class:

py
from typing import Generic, Literal, TypeVar

T = TypeVar("T")

class GenericBase(Generic[T]):
    def __add__(self, other: object) -> Literal["left"]:
        return "left"

class GenericChild(GenericBase[T]):
    def __radd__(self, other: object) -> Literal["right"]:
        return "right"

def add_generic(left: GenericBase[int], right: GenericChild[str]) -> Literal["left", "right"]:
    reveal_type(left + right)  # revealed: Literal["right", "left"]
    return left + right

Class objects use their metaclasses for reflected precedence

The runtime classes of class objects are their metaclasses. If the right operand's metaclass is a strict subclass of the left operand's metaclass, its reflected method takes precedence:

py
from typing import Literal

class LeftMeta(type):
    def __add__(cls, other: object) -> Literal["left"]:
        return "left"

class RightMeta(LeftMeta):
    def __radd__(cls, other: object) -> Literal["right"]:
        return "right"

class A(metaclass=LeftMeta): ...
class B(metaclass=RightMeta): ...

reveal_type(A + B)  # revealed: Literal["right"]

TypeVars and NewTypes do not have an exact runtime class

The upper bound of a TypeVar is not necessarily its runtime class, so it cannot decide definitively whether the right-hand operand's reflected method takes precedence. A NewType constructor also returns its argument unchanged, so an inhabitant can have a runtime class below the NewType's base:

py
from typing import Literal, NewType, TypeVar

class Base:
    def __add__(self, other: object) -> Literal["base"]:
        return "base"

class Child(Base):
    def __radd__(self, other: object) -> Literal["child"]:
        return "child"

T = TypeVar("T", bound=Base)

def add_child(left: T) -> Literal["base", "child"]:
    reveal_type(left + Child())  # revealed: Literal["child", "base"]
    return left + Child()

U = TypeVar("U", bound=Child)

def add_typevar(left: Base, right: U) -> Literal["base", "child"]:
    reveal_type(left + right)  # revealed: Literal["child", "base"]
    return left + right

NewChild = NewType("NewChild", Child)

def add_newtype(left: Base, right: NewChild) -> Literal["base", "child"]:
    reveal_type(left + right)  # revealed: Literal["child", "base"]
    return left + right

Reflected precedence 2

If the right-hand operand is a subtype of the left-hand operand, but does not override the reflected method, the left-hand operand's non-reflected method still takes precedence:

py
class A:
    def __add__(self, other) -> str:
        return "foo"

    def __radd__(self, other) -> int:
        return 42

class B(A): ...

reveal_type(A() + B())  # revealed: str

Only reflected supported

For example, at runtime, (1).__add__(1.2) is NotImplemented, but (1.2).__radd__(1) == 2.2, meaning that 1 + 1.2 succeeds at runtime (producing 2.2). The runtime tries the second one only if the first one returns NotImplemented to signal failure.

Typeshed and other stubs annotate dunder-method calls that would return NotImplemented as being "illegal" calls. int.__add__ is annotated as only "accepting" ints, even though it strictly-speaking "accepts" any other object without raising an exception -- it will simply return NotImplemented, allowing the runtime to try the __radd__ method of the right-hand operand as well.

py
class A:
    def __sub__(self, other: "A") -> "A":
        return A()

class B:
    def __rsub__(self, other: A) -> "B":
        return B()

reveal_type(A() - B())  # revealed: B

Callable instances as dunders

Believe it or not, this is supported at runtime:

py
class A:
    def __call__(self, other) -> int:
        return 42

class B:
    __add__ = A()

reveal_type(B() + B())  # revealed: int

We also infer int if the callable is declared:

py
class B2:
    __add__: A = A()

reveal_type(B2() + B2())  # revealed: int

Integration test: numbers from typeshed

We get less precise results from binary operations on float/complex literals due to the special case for annotations of float or complex, which applies also to return annotations for typeshed dunder methods. Perhaps we could have a special-case on the special-case, to exclude these typeshed return annotations from the widening, and preserve a bit more precision here?

py
reveal_type(3j + 3.14)  # revealed: int | float | complex
reveal_type(4.2 + 42)  # revealed: int | float
reveal_type(3j + 3)  # revealed: int | float | complex
reveal_type(3.14 + 3j)  # revealed: int | float | complex
reveal_type(42 + 4.2)  # revealed: int | float
reveal_type(3 + 3j)  # revealed: int | float | complex

def _(x: bool, y: int):
    reveal_type(x + y)  # revealed: int
    reveal_type(4.2 + x)  # revealed: int | float
    reveal_type(y + 4.12)  # revealed: int | float

With literal types

When we have a literal type for one operand, we're able to fall back to the instance handling for its instance super-type.

py
class A:
    def __add__(self, other) -> "A":
        return self

    def __radd__(self, other) -> "A":
        return self

reveal_type(A() + 1)  # revealed: A
reveal_type(1 + A())  # revealed: A

reveal_type(A() + "foo")  # revealed: A
reveal_type("foo" + A())  # revealed: A

reveal_type(A() + b"foo")  # revealed: A
reveal_type(b"foo" + A())  # revealed: A

reveal_type(A() + ())  # revealed: A
reveal_type(() + A())  # revealed: A

literal_string_instance = "foo" * 1_000_000_000
# the test is not testing what it's meant to be testing if this isn't a `LiteralString`:
reveal_type(literal_string_instance)  # revealed: LiteralString

reveal_type(A() + literal_string_instance)  # revealed: A
reveal_type(literal_string_instance + A())  # revealed: A

Operations involving instances of classes inheriting from Any

Any and Unknown represent a set of possible runtime objects, wherein the bounds of the set are unknown. Whether the left-hand operand's dunder or the right-hand operand's reflected dunder depends on whether the right-hand operand is an instance of a class that is a subclass of the left-hand operand's class and overrides the reflected dunder. In the following example, because of the unknowable nature of Any/Unknown, we must consider both possibilities: Any/Unknown might resolve to an unknown third class that inherits from X and overrides __radd__; but it also might not. Thus, the correct answer here for the reveal_type is int | Unknown.

py
from does_not_exist import Foo  # error: [unresolved-import]

reveal_type(Foo)  # revealed: Unknown

class X:
    def __add__(self, other: object) -> int:
        return 42

class Y(Foo): ...

# TODO: Should be `int | Unknown`; see above discussion.
reveal_type(X() + Y())  # revealed: int

Operations involving types with invalid __bool__ methods

py
class NotBoolable:
    __bool__: int = 3

a = NotBoolable()

# snapshot: unsupported-bool-conversion
10 and a and True
snapshot
error[unsupported-bool-conversion]: Boolean conversion is not supported for type `NotBoolable`
 --> src/mdtest_snippet.py:7:8
  |
7 | 10 and a and True
  |        ^
  |
info: `__bool__` on `NotBoolable` must be callable

Operations on class objects

When operating on class objects, the corresponding dunder methods are looked up on the metaclass.

py
from __future__ import annotations

class Meta(type):
    def __add__(self, other: Meta) -> int:
        return 1

    def __lt__(self, other: Meta) -> bool:
        return True

    def __getitem__(self, key: int) -> str:
        return "a"

class A(metaclass=Meta): ...
class B(metaclass=Meta): ...

reveal_type(A + B)  # revealed: int
# error: [unsupported-operator] "Operator `-` is not supported between objects of type `<class 'A'>` and `<class 'B'>`"
reveal_type(A - B)  # revealed: Unknown

reveal_type(A < B)  # revealed: bool
reveal_type(A > B)  # revealed: bool

# error: [unsupported-operator] "Operator `<=` is not supported between objects of type `<class 'A'>` and `<class 'B'>`"
reveal_type(A <= B)  # revealed: Unknown

reveal_type(A[0])  # revealed: str

Unsupported

Dunder as instance attribute

The magic method must exist on the class, not just on the instance:

py
def add_impl(self, other) -> int:
    return 1

class A:
    def __init__(self):
        self.__add__ = add_impl

# error: [unsupported-operator] "Operator `+` is not supported between two objects of type `A`"
# revealed: Unknown
reveal_type(A() + A())

Missing dunder

py
class A: ...

# error: [unsupported-operator]
# revealed: Unknown
reveal_type(A() + A())

Wrong position

A left-hand dunder method doesn't apply for the right-hand operand, or vice versa:

py
class A:
    def __add__(self, other) -> int:
        return 1

class B:
    def __radd__(self, other) -> int:
        return 1

class C: ...

# error: [unsupported-operator]
# revealed: Unknown
reveal_type(C() + A())

# error: [unsupported-operator]
# revealed: Unknown
reveal_type(B() + C())

Reflected dunder is not tried between two objects of the same type

For the specific case where the left-hand operand is the exact same type as the right-hand operand, the reflected dunder of the right-hand operand is not tried; the runtime short-circuits after trying the unreflected dunder of the left-hand operand. For context, see this mailing list discussion.

py
class Foo:
    def __radd__(self, other: "Foo") -> "Foo":
        return self

# error: [unsupported-operator]
# revealed: Unknown
reveal_type(Foo() + Foo())

Wrong type

TODO: check signature and error if other is the wrong type