packages/shared-skills/skills/programming/references/python/README.md
Modern Python. Type-strict, stack-first, async-correct.
The type checker is your compiler. Make illegal states unrepresentable. Parse at boundaries. Own resources explicitly. Every function has a contract; the type system enforces it.
These are deliberate project choices. Violations are always wrong, not "style preferences".
| Category | Use | Never |
|---|---|---|
| Package manager | uv | pip, poetry, conda, pipenv |
| Type checker | basedpyright (typeCheckingMode = "all") | pyright, mypy |
| Linter + formatter | ruff (select = ["ALL"]) | flake8, black, isort, autopep8 |
| Async runtime | anyio | import asyncio |
| Data | polars + duckdb + numpy | pandas |
| Web framework | FastAPI + Pydantic v2 | Flask, Django REST |
| ORM | SQLAlchemy 2.x async | Django ORM, Tortoise |
| HTTP client | httpx2 | requests, aiohttp, httpx |
| Testing | pytest | unittest |
| CLI | typer + rich | argparse, click, fire |
@dataclass(frozen=True, slots=True). Pydantic: model_config = ConfigDict(frozen=True). Mutable only when mutation is the documented purpose.UserId = NewType("UserId", int). Never pass raw int where a branded type exists.match only for variants, if only for booleans — NEVER use if/elif/else to discriminate on type (isinstance), enum value, or literal variant. match/case is mandatory for these — non-negotiable. ALWAYS end with case unreachable: assert_never(unreachable) — bare case _: pass and case _: raise ValueError are banned (they silently swallow new variants). if/else is fine only for boolean expressions, range checks, and predicate calls that aren't variant discrimination. See "Why if/elif on variants is banned" below for examples.typing.Protocol for interfaces. ABC only when you need shared method implementation.TypedDict, dataclass, or Pydantic model. Internal scratch dicts are fine.raise ValueError("something") with a bare string. Use union returns when the caller is within 1-2 call levels and must handle the outcome (repository → service). Use exceptions when the error should propagate up many layers to a boundary handler (service → HTTP handler).Final. Mutable module globals are a code smell.-> X | None. Never return None from a function whose signature omits it..close().object erases all structural information (zero callable attributes, zero narrowing). Use Protocol (structural typing), TypeVar (generic pass-through), explicit union (known variants), or TypedDict (dict shapes).cast() is banned. Redesign the types.except Exception and except BaseException are banned. Catch the specific exception you expect. A broad catch swallows bugs you need to see — KeyError, AttributeError, TypeError all vanish silently. If you genuinely need a catch-all at a top-level boundary (CLI entry, HTTP handler), use # noqa: BROAD_EXCEPT_OK and log + re-raise.basedpyright in typeCheckingMode = "all". Every public function has full annotations. Internal helpers: annotate return type; parameter types may be inferred.ruff with select = ["ALL"]. Override specific rules per project in pyproject.toml, never globally disable the strict baseline.docstring unless its name + signature makes it completely obvious (e.g. def full_name(first: str, last: str) -> str:).X | Y union syntax (PEP 604), never Union[X, Y] or Optional[X].object is bannedobject pretends to be safe ("it's the top type!") but gives zero narrowing and zero attributes. Even Any is more honest — it admits the boundary is untyped.
# BANNED
def process(data: object) -> object: ...
def store(items: list[object]) -> None: ...
results: dict[str, object] = {}
# GOOD — Protocol for structural typing
class Serializable(Protocol):
def serialize(self) -> bytes: ...
def process(data: Serializable) -> ProcessResult: ...
# GOOD — TypeVar for generic pass-through
def identity[T](x: T) -> T: ...
def first[T](items: Sequence[T]) -> T: ...
# GOOD — explicit union for known variants
def parse(raw: str | bytes) -> Document: ...
if/elif on variants is bannedif/elif/else chains on type, enum, or literal values lose compile-time exhaustiveness. When a new variant is added, nothing warns you. match/case + assert_never does.
# BANNED — if/elif for type discrimination
if isinstance(event, Click):
handle_click(event.x, event.y)
elif isinstance(event, Scroll):
handle_scroll(event.delta)
else:
raise ValueError(f"Unknown: {event}") # runtime bomb
# BANNED — if/elif for enum discrimination
if status == Status.PENDING:
start_review()
elif status == Status.ACTIVE:
continue_processing()
elif status == Status.CLOSED:
archive()
# BANNED — non-exhaustive match (swallows new variants)
match event:
case Click(x, y): handle_click(x, y)
case _: pass
# GOOD — exhaustive match with assert_never
match event:
case Click(x=x, y=y):
handle_click(x, y)
case Scroll(delta=delta):
handle_scroll(delta)
case unreachable:
assert_never(unreachable)
# GOOD — enum match
match status:
case Status.PENDING: start_review()
case Status.ACTIVE: continue_processing()
case Status.CLOSED: archive()
case unreachable: assert_never(unreachable)
if/else is fine for boolean conditions and range checks — things that aren't variant discrimination:
# FINE — boolean, not variant
if age >= 18:
grant_access()
else:
deny_access()
except is bannedexcept Exception catches every non-system exception — KeyError, TypeError, AttributeError, ValueError all vanish. You lose the stack trace that would have told you exactly what went wrong. The fix is always to name the exception you expect.
# BANNED — swallows bugs
try:
result = api.fetch(url)
except Exception as e:
logger.error(e)
return None
# BANNED — catch-and-ignore
try:
parse(data)
except Exception:
pass
# GOOD — catch what you expect
try:
result = api.fetch(url)
except httpx.HTTPStatusError as e:
logger.error("API %d: %s", e.response.status_code, e.request.url)
return None
except httpx.ConnectError:
raise ServiceUnavailableError(service="api") from None
# GOOD — top-level boundary (only place broad catch is acceptable)
def main() -> int: # noqa: BROAD_EXCEPT_OK
try:
return run()
except Exception:
logger.exception("unhandled error")
return 1
import asyncio is BANNED. Use import anyio.anyio.create_task_group. Never fire-and-forget with asyncio.create_task.anyio.CapacityLimiter (not asyncio.Semaphore).async-anyio.md when writing async code for the full pattern library.All model fields carry type annotations. No Any, no untyped dicts in public APIs.
Use polars + duckdb for data. pandas is never the right answer in this stack.
| Situation | Use |
|---|---|
| User input, API request/response | Pydantic BaseModel (frozen=True) |
| Internal value object (no I/O) | @dataclass(frozen=True, slots=True) |
| Function with multiple outcomes | Union of frozen dataclasses + match |
Dict shape for JSON compat / **kwargs | TypedDict |
| Fixed constants | StrEnum / IntEnum |
| Distinct primitive (UserId vs MovieId) | NewType |
| Contract / capability | Protocol |
| Contract + shared implementation | ABC |
| ORM model (SQLAlchemy) | Mapped[] — inherently mutable, # noqa: MUTABLE_OK |
| Config from env vars | pydantic-settings BaseSettings |
The one rule: data crosses trust boundary → Pydantic. Everything else → dataclass.
Load data-modeling.md for the full decision flowchart and comparison matrix.
Mapped[] requires mutation. Use # noqa: MUTABLE_OK.If you need # noqa: MUTABLE_OK, the class docstring must say why mutation is required.
Canonical defaults (override only if pyproject.toml explicitly picks something else):
| Domain | Library | Reason |
|---|---|---|
| CLI | typer | Type-annotated CLI from function sigs |
| Pretty output | rich | Tables, progress, tracebacks, markdown |
| HTTP client | httpx2 | Next-gen HTTP client (Pydantic stewardship), HTTP/2, brotli+zstd. Always httpx2[http2,brotli,zstd]. See httpx2-optimization.md |
| Validation | pydantic v2 | Fast native validator, JSON Schema |
| Web API | fastapi | Async, Pydantic-native, OpenAPI |
| ORM | sqlalchemy 2.x async | Mapped[] types, async sessions |
| DB driver (Postgres) | asyncpg (via SQLAlchemy) | Fastest PG driver |
| AI agents | pydantic-ai | Typed deps, structured output |
| TUI | textual | Rich-based, CSS layout, widgets |
| Logging | rich.logging.RichHandler | Pretty; swap to structlog in prod |
Scaffold a new project with all strict defaults pre-configured:
uv run ../../scripts/python/new-project.py myproject
uv run ../../scripts/python/new-project.py myproject --path ./workspace
uv run ../../scripts/python/new-project.py myproject --lib # publishable library
Creates via uv init, then injects basedpyright typeCheckingMode = "all" + ruff select = ["ALL"] + pytest strict. Cross-platform (macOS, Linux, Windows).
For manual setup: uv init --app myproject, then load pyproject-strict.md.
Every .py script — even throwaway — MUST use PEP 723 inline metadata with the # ─── How to run ─── comment block. No venv, no requirements.txt. The script IS the environment spec. A script without the usage comment block is incomplete.
Scaffold with: uv run ../../scripts/python/new-script.py <name> --deps "httpx2[http2,brotli,zstd]" (writes to temp dir by default, --output for specific path).
Load one-liners.md for full patterns, examples, and anti-patterns.
Load on demand — not all at once.
| Need | Load |
|---|---|
| Full pyproject.toml config | pyproject-strict.md |
| Type patterns (NewType, Final, enums, narrowing) | type-patterns.md |
| Data modeling (container choice, frozen, parse-don't-validate) | data-modeling.md |
| Error handling (typed errors, union returns, exhaustive match) | error-handling.md |
| Async patterns (anyio) | async-anyio.md |
| Data processing (polars / duckdb) | data-processing.md |
| FastAPI + SQLAlchemy stack | fastapi-stack.md |
| Library decision tree | libraries.md |
| httpx2 optimization (MUST load for any network code) | httpx2-optimization.md |
| orjson (when JSON is in the hot path; FastAPI/Pydantic v2 integration) | orjson-stack.md |
| One-liner scripts (PEP 723) | one-liners.md |
| PydanticAI agents | pydantic-ai.md |
| Textual TUI | textual-tui.md |
Every outgoing HTTP call MUST use httpx2 (httpx2[http2,brotli,zstd]). Never requests, never aiohttp, never the original httpx.
ALL optimizations are ON by default — not optional, not progressive, not "nice to have". A bare httpx2.AsyncClient() is a bug — treat it like a lint violation. The correct way is the factory pattern in httpx2-optimization.md with: HTTP/2 enabled, tuned connection pool (200/40/30s), split timeouts (5/30/10/10), transport retries (3), TCP_NODELAY, follow_redirects, and event hooks for observability.
When writing or reviewing ANY network code, ALWAYS load httpx2-optimization.md and use the factory pattern verbatim. No exceptions.
Violations caught by ../../scripts/python/check-no-excuse-rules.py. Run after every edit session.
| Rule ID | Catches | Opt-out |
|---|---|---|
cast-any | cast(Any, ...) | None — redesign types |
type-ignore | # type: ignore | None — fix the type |
pyright-ignore | # pyright: ignore | None — fix the type |
bare-except | except: with no class | None — name the exception |
silent-except | except X: pass / except X: ... | None — handle or re-raise |
no-asyncio | import asyncio | # noqa: ANYIO_OK |
no-pandas | import pandas | # noqa: PANDAS_OK |
mutable-dataclass | @dataclass without frozen=True | # noqa: MUTABLE_OK |
missing-slots | @dataclass without slots=True | # noqa: SLOTS_OK |
raw-dict-return | -> dict in function return type | # noqa: DICT_OK |
missing-assert-never | match block without assert_never default | # noqa: MATCH_OK |
generic-exception | raise ValueError("...") / raise TypeError("...") with bare string | # noqa: GENERIC_ERR_OK |
no-object | object used as type annotation (param, return, generic arg) | # noqa: OBJECT_OK |
if-elif-on-variant | if isinstance()/if x == Enum.V chain that should be match/case | # noqa: IF_VARIANT_OK |
oversized-module | File exceeds 250 pure LOC (non-blank, non-comment) | # noqa: SIZE_OK |
broad-except | except Exception / except BaseException (too broad) | # noqa: BROAD_EXCEPT_OK |
Fix every violation before declaring work done. basedpyright + ruff strict config catches the rest.
Tests are strict too, with these exceptions (already configured in pyproject.toml per-file-ignores):
| In tests you may | Why |
|---|---|
Use assert | That's how pytest works (S101 ignored) |
| Use magic numbers | Test data (PLR2004 ignored) |
Access _private members | Testing internals (SLF001 ignored) |
| Skip docstrings | Test names are the docs (D ignored) |
| Have unused function args | Fixtures (ARG ignored) |
Tests still follow the iron list — frozen dataclasses, typed errors, exhaustive match. If test fixtures need mutable state, use # noqa: MUTABLE_OK on the fixture class.
When editing an existing file that doesn't follow these rules: write new code in strict style, don't refactor existing code in the same change. Mixing feature work with style migration makes reviews harder and bugs likelier.
This skill activates whenever you are writing or modifying any .py file. Even one-off scripts get the strict treatment — that is the whole point of PEP 723 + uv: production hygiene with throwaway ergonomics.