limitations/encoding.md
str.encode() / bytes.decode()Monty implements a fixed, small set of text codecs rather than the full
codecs/encodings registry CPython ships. The str(b, encoding, errors)
and bytes(s, encoding, errors) constructor forms route through the same
registry, so everything below applies to them too.
utf-8asciiutf-16, utf-16-le, utf-16-beutf-32, utf-32-le, utf-32-beNames are normalized like CPython (case-insensitive; runs of spaces/hyphens
collapse to _) and each codec's CPython aliases are recognized (utf8,
u16, 646, us-ascii, ...). Any other encoding name raises
LookupError: unknown encoding: {name} — including names CPython recognizes
(latin-1, cp1252, iso-8859-1, big5, ...).
utf-16 / utf-32CPython's bare utf-16/utf-32 codecs use the platform's native byte
order: encode writes a BOM in native order, and BOM-less input decodes as
native order. Monty always uses little-endian for both, so behavior is
identical on every little-endian host (all platforms Monty CI covers) but
diverges on big-endian hosts. Input with a BOM decodes identically
everywhere (the BOM's order wins).
bytes.decode(..., errors='surrogateescape') raises
NotImplementedError when a byte actually needs handling: CPython maps
undecodable bytes to lone surrogates (U+DC80–U+DCFF), which Monty's
strict-UTF-8 strings cannot contain.errors='surrogatepass' on decode raises NotImplementedError in the
cases where CPython would produce a lone surrogate (a CESU-8 surrogate
triple in UTF-8 input; a lone surrogate unit/code point in UTF-16/32
input). For any other invalid input it re-raises the strict
UnicodeDecodeError, matching CPython.codecs.register_error do not exist
(there is no codecs module); any name outside CPython's built-in set
raises LookupError: unknown error handler name '{name}'.namereplace output for recently-added code points is subject to the
Unicode version skew described in unicodedata.md.UnicodeEncodeError / UnicodeDecodeErrorInside the sandbox both are message-only, like every other Monty
exception — see exceptions.md.
CPython's encoding/object/start/end/reason attributes are not
exposed to sandboxed code, and the in-sandbox constructor accepts only a
single message argument.
On the host, codec errors carry the structured constructor fields, so
pydantic_monty's .exception() rebuilds a real UnicodeDecodeError /
UnicodeEncodeError with all five CPython attributes. The host falls back
to a plain ValueError carrying the formatted message (both are caught by
except ValueError:) in two cases:
raise UnicodeDecodeError('msg')), where no structured fields exist.The structured fields only travel with a raised exception that escapes the
sandbox. A codec exception handled as a value — caught in the sandbox and
then returned as the run result, or passed to an external function — crosses
the boundary as a message-only exception object, so the host sees the
ValueError fallback for it even though the same exception raised out of
the sandbox would rebuild the real type.
The JavaScript package does not reconstruct host-side exception instances,
so this applies to pydantic_monty only.