docs/releasenotes/version17.md
% SPDX-FileCopyrightText: 2022 James R. Barlow % SPDX-License-Identifier: CC-BY-SA-4.0
Enhancements
--max-ocr-image-mpixels downsamples the image sent to OCR when a page
exceeds the given size, which bounds the largest consumer of memory. The
visible page is never downsampled, so output appearance is unaffected in every
mode; what it trades is OCR accuracy on very high resolution scans. A 34
megapixel page that peaks at 492 MB peaks at 325 MB under
--max-ocr-image-mpixels 8, and recognizes the same text. See the "Memory"
section of the performance documentation for how to size a memory limit.watcher.py (the watcher extra) gained a configurable output layout and
conflict policy. These are watcher-only changes; they do not affect the
ocrmypdf library API.
OCR_OUTPUT_STRUCTURE setting (--output-structure): FLAT
(default, all outputs directly in the destination directory),
YEAR_MONTH ({destination}/{year}/{month}/{filename}, same layout as
the old OCR_OUTPUT_DIRECTORY_YEAR_MONTH=1), or HIERARCHY, which
mirrors the input directory tree under the destination, e.g.
input/a/b/c.pdf → output/a/b/c.pdf.OCR_ON_CONFLICT setting (--on-conflict) controls what happens
when the intended destination file already exists: SUFFIX (default)
writes name (1).pdf, name (2).pdf, ... in the OS style; SKIP logs
and leaves the file unprocessed; OVERWRITE is the old behavior.
Behavior change: the default is now SUFFIX, so existing output
files are no longer silently overwritten.OCR_OUTPUT_DIRECTORY and to the
archive directory used by OCR_ON_SUCCESS_ARCHIVE; previously the
archive directory was always flat and silently overwrote on a name
collision.<>:"/\|?* and control
characters) are replaced with _, trailing dots/spaces are stripped,
and reserved DOS device names (CON, PRN, AUX, NUL, COM1-9,
LPT1-9) are prefixed with _.OCR_OUTPUT_DIRECTORY_YEAR_MONTH is now deprecated in favor of
OCR_OUTPUT_STRUCTURE=YEAR_MONTH. It is still honored and logs a
deprecation warning; if both are set, OCR_OUTPUT_STRUCTURE wins.Performance
pi-heif already required Pillow 11.1, so the effective floor barely moves).ocrmypdf.ocr(), so a second
call in another thread had to wait for the first to finish. Plugin state is
now guarded by a readers-writer lock: installing a plugin set takes it
exclusively, and a job holds it shared for its run. An in-flight job therefore
cannot have the plugin infrastructure it depends on replaced underneath it,
while jobs that are past installation proceed concurrently.ocrmypdf.ocr() call
re-executed plugin modules given as file paths and rebound them in
sys.modules. Plugins must not rely on being re-executed for each job, and
must not store per-job state on the plugin manager, which concurrent jobs
requesting the same plugin set now share./FlateDecode with a PNG predictor already holds exactly
what a PNG IDAT chunk holds, so it is now repackaged as a PNG directly
instead of being decoded to a bitmap and re-encoded. On a 6-page document
containing one 9000x9000 image, the optimization step went from 3.4s to 0.6s,
and to 0.04s together with the JPEG change below; total runtime went from
10.0s to 7.0s. Output is unchanged: the compressed
data is reused verbatim. Images that are not in a directly repackageable form
still take the previous path.--optimize 2). We now decode only on code paths that
use the decoded JPEG. Output is unchanged./Filter) no longer produces a spurious
"could not be processed by the optimizer" warning. Such an image raised
IndexError internally, which the optimizer's best-effort handler caught and
reported as a warning; it is now recognized and skipped quietly.Executor instances. Executor.pool_lock is retained but no longer acquired,
and is deprecated; it will be removed in a future major release. The invariant
it protected - that only one progress bar renders on the shared console - is
now enforced by the progress bar, which disables itself if another bar already
owns the console.jobs=M may now spawn up to
N*M workers, where previously they were serialized to M. Size jobs
accordingly.max_image_mpixels. Pillow's decompression-bomb limit is interpreter-global
and the last job to set it wins. Use separate processes to run jobs with
differing configurations.Fixes
to_pil() lets
Pillow alias PDFium's bitmap buffer for some formats -- grayscale renders
among them, which is every mono and grayscale page -- and the buffer was freed
immediately afterwards, leaving Pillow reading memory PDFium had released.[WinError 2] The system cannot find the file specified warnings while it searches for Ghostscript and Tesseract
(#1671). These
messages came from probing registry keys that simply don't exist when the
programs were installed by a package manager such as Scoop, or not installed
at all. Since the search then continues elsewhere and usually succeeds, these
failures are normal, and are now logged at debug level, naming the location
that was searched. If a program genuinely cannot be found, OCRmyPDF still
reports that as an error.PROGRAMFILES environment variable pointed to
a folder that does not exist.--mode strip failed to remove OCR text layers that OCRmyPDF itself
produced ({issue}1730). OCRmyPDF grafts its text layer as a Form XObject
and stripping only examined the page content stream, so the invisible text
was never found. The same flaw made --mode redo stack a second text layer
on top of the old one instead of replacing it. Stripping now descends into
Form XObjects. Thanks @Anai-Guo ({issue}1732)./Resources inherited from an ancestor
/Pages node, rather than only looking at the page's own resources.clean_final on an existing options object in the Python API no
longer leaves clean unset. --clean-final implies --clean, but the rule
lived in a field validator that only ran while the options object was being
constructed, so assigning to the attribute afterwards silently skipped it.--jobs 999 now
reports --jobs: Input should be less than or equal to 256. The set of
accepted values is unchanged.The watcher.py watched-folder helper (the watcher extra) has been
modernized and security-hardened:
watchfiles instead of watchdog. Installing
ocrmypdf[watcher] now pulls in watchfiles; native OS filesystem
notifications are used by default, with OCR_USE_POLLING=1 to force
polling.sys.path,
the virtual environment, site-packages, or $PATH), if
OCR_JSON_SETTINGS points at a file inside a data directory or one that
is group/world-writable, or if it specifies a plugin located inside a
data directory. It also refuses to run when the output or archive
directory is the input directory or a subdirectory of it, which would
otherwise cause OCRmyPDF output to be reprocessed in an endless loop.1715). pikepdf.PasswordError does not derive from
pikepdf.PdfError, so it escaped the handler that waits for a file to be
fully written and tore down the watch loop, leaving files that arrived
afterwards unprocessed. Encrypted files are now logged and skipped
immediately — no amount of retrying will supply the password. More
generally, no per-file error can stop the watcher now: failures are
logged and watching continues. Thanks @christophdb for the report and a
fix ({issue}1716).See the "Watcher security model" section of the batch processing documentation for details. Existing deployments where the data directories are kept separate from the application are unaffected; deployments that co-located data with the interpreter or its environment will need to relocate one or the other.
Ghostscript 10.7.0 and later are no longer treated as affected by the JPEG
passthrough truncation bug, which Ghostscript fixed in 10.07.0
({issue}1726). The version check had no upper bound, so users on a fixed
Ghostscript still saw the "JPEG encoding errors" warning and, worse, silently
had every JPEG lossily re-encoded at --optimize 1 (the default) to work
around a bug their Ghostscript did not have. Thanks @zuentec-droid for the
detailed measurements and upstream analysis.
The same JPEG re-encoding workaround no longer applies when Ghostscript did
not produce the file at all. It was previously triggered by the mere presence
of an affected Ghostscript, so --output-type pdf and files converted by the
speculative PDF/A path — neither of which runs Ghostscript — paid the quality
loss for nothing.
1722). Previously a document
in, say, Cherokee or Vai was rendered with the glyphless fallback font even
though the matching font was installed — a common situation on macOS, which
ships around a hundred script-specific Noto faces. When the named fonts
cannot cover a word, OCRmyPDF now searches the installed fonts for one that
can.font-noto) that does not exist ({issue}1722). Homebrew has no
single Noto package; each family is a separate cask. The Fedora package name
was also corrected to google-noto-fonts-all.GlyphSearchingFontProvider
protocol to participate in coverage-based font search.--jpeg-quality/--jpg-quality having no effect on the CLI: the
value was silently dropped before reaching the optimizer, which then
always used its own built-in default JPEG quality regardless of what was
requested ({issue}1723). The same bug affected the Python API's
jpg_quality parameter. ocrmypdf.ocr() now accepts jpeg_quality
(matching the CLI flag name) as the canonical parameter; jpg_quality
still works but is deprecated./Resources,
/XObject, and /FontDescriptor entries, which previously crashed
ocrmypdf.ocr() with AttributeError/TypeError/ValueError on
otherwise-processable files, both during PDF/A font scanning and general
image scanning ({issue}1713). Thanks @mvanhorn for the initial fix.--tesseract-pagesegmode help text to point to
tesseract --help-extra, since Tesseract 5.5.2 moved the page segmentation
mode documentation there from tesseract --help. Thanks @sokai.--check-untyped-defs is now enabled, and the mypy pre-commit hook is now
blocking rather than advisory), fixing several latent edge-case bugs
surfaced along the way.Path.--output-type auto (the default) again produces PDF/A whenever it can,
matching OCRmyPDF 16's "PDF/A by default" behavior. It first tries the fast
Ghostscript-free conversion (validated by veraPDF when available) and now
falls back to Ghostscript when that cannot produce PDF/A, only emitting a
regular PDF when even Ghostscript cannot safely convert (for example, an
input with non-embedded CID/CJK fonts, per {issue}1561). A consequence is
that the default path may once again invoke Ghostscript, which is slower and
may transcode images; use --output-type pdf to skip PDF/A conversion
entirely.1561). PDF/A requires all fonts to be embedded, so
Ghostscript substitutes and re-embeds non-embedded CID fonts — such as the OCR
text layer Adobe Acrobat adds to scanned CJK documents — which mangles the
text and destroys searchability. OCRmyPDF now detects non-embedded CID fonts
before conversion: with --output-type auto (the default) it produces a
regular PDF and preserves the existing text layer, and with an explicit
--output-type pdfa* it stops with an error rather than emit corrupted
output. Use --output-type pdf to keep the text layer, or --force-ocr to
rebuild it with embedded fonts.ocrmypdf input.pdf -) is now
protected against corruption at the operating system level. Previously
OCRmyPDF relied on no in-process code — third-party libraries, plugins, or
stray print() calls — ever writing to stdout; a single accidental write
would silently corrupt the PDF. The command line program now saves the real
stdout at startup, before plugins are loaded or any worker process/thread is
started, and redirects file descriptor 1 to stderr, so that only OCRmyPDF's
final PDF output can reach stdout. A consequence is that a plugin which
intentionally prints to stdout will have that output redirected to stderr.ocrmypdf.configure_stdout_protection,
which installs this same protection. Like {func}ocrmypdf.configure_logging,
it is optional and intended for callers that want command-line-like behavior;
applications that manage their own standard output should not call it.UnicodeDecodeError when processing a PDF whose
/DocumentInfo dictionary contains a /Name key encoded in Latin-1 (or
another non-UTF-8 encoding), such as /Saks#e5r. repair_docinfo_nuls now
treats such a block as malformed, logs a message, and continues instead of
crashing the pipeline ({issue}1540). Current pikepdf releases tolerate these
keys by surrogate-escaping them, but older versions raised while iterating the
dictionary.--redo-ocr ({issue}1662). Since
v16.4.3, OCRmyPDF forced pdfminer's read buffer to 256 MiB to work around a
pdfminer bug that mishandled tokens split across the buffer boundary
({issue}1361). On Windows, CPython's BufferedReader.read() eagerly
allocates a buffer of the requested size on every read, so the oversized
buffer made each of pdfminer's thousands of reads cost tens of milliseconds
(this allocation is lazy, and effectively free, on Linux). The underlying
pdfminer bug was fixed upstream in pdfminer.six 20250327
(#1030), with a
follow-up for tokens split across streams in 20260107
(#1158), so the
workaround has been removed and the minimum pdfminer.six version raised to
20260107.NotoSansArabic[wdth,wght].ttf, the form shipped by Homebrew casks
and current Google Fonts releases. Previously only static -Regular.ttf/.otf
files were matched, so users who had installed the correct Noto font still got
the glyphless fallback and a "No font found" warning ({issue}1652).app, uid/gid 1000) by default
rather than as root, as a defense-in-depth measure. If you bind-mount a
directory for input and output, you may now need to add a --user argument so
the container can write to it; the correct value differs for rootless Docker,
Podman, and rootful Docker, and is described in the Docker documentation.
Piping the input and output through stdin/stdout still works with no
permission setup./data, so files in
a directory mounted there can be given as relative paths without an explicit
--workdir.alex-p/tesseract-ocr5 PPA, and the base images
were updated to Ubuntu 26.04 and Alpine 3.24.-v 1) ({issue}846).--pdfa-image-compression=auto (the default) now selects lossless image
compression at -O0 so Ghostscript no longer transcodes lossless images to
JPEG during PDF/A generation. At -O1 and above, auto continues to defer
to Ghostscript's heuristic, which may recompress images lossily. -O1 (the
default level) is kept as a historical exception because coercing it to
lossless can substantially bloat output; users who want guaranteed lossless
image handling should pass --pdfa-image-compression=lossless or use -O0
({issue}1124).--pdfa-image-compression=lossless now passes existing JPEG images through
unchanged rather than re-encoding them with a lossless codec. Re-encoding an
already-lossy JPEG losslessly cannot recover quality and only inflates the
file, so JPEGs are preserved while non-JPEG images are encoded losslessly./MediaBox, /CropBox, /TrimBox, /ArtBox, /BleedBox) in its
input, following the PDF 2.0 specification. Coordinates written in invalid
exponential notation are reinterpreted ({issue}1398); rectangles whose
corners are given in reversed order are normalized, which previously crashed
with NegativeDimensionError ({issue}1526); and a crop/trim/art/bleed box
that falls outside the MediaBox is clamped to their intersection, or discarded
when that intersection is empty, which previously produced an output with a
zero-height effective page that some viewers refused to open ({issue}1400).
When a box is discarded, clamped, or reinterpreted, OCRmyPDF logs a warning
recommending visual inspection of the output. Thanks @ajdlinux for the initial
fix in PR #1691./Root/PieceInfo/SearchIndex) from its output. This proprietary index,
produced by Acrobat's "Embed Index" feature, is read only by Adobe Acrobat;
other viewers ignore it and search the text on the fly. Because any change to
a PDF invalidates the index, retaining it after OCRmyPDF rewrites the document
would leave a stale index that returns incorrect search results in Acrobat.
Modern viewers rebuild a search index on demand, so there is no loss of
search capability./Thumb image XObject on a page) from its output. OCRmyPDF alters page
appearance (deskew, clean, rasterize, re-render) and plugins may edit pages
arbitrarily, so a retained thumbnail would be stale and no longer match its
page. Embedded thumbnails are a navigation aid that modern viewers generate
on demand, so there is no loss of functionality.1688pngmonod
(error-diffusion) instead of pngmono (ordered dithering). It produces
better input for OCR on faint or anti-aliased scans at negligible cost and
no change to output file size, since the rasterized image is an
intermediate that is discarded after OCR.-dTextAlphaBits=4 -dGraphicsAlphaBits=4) for the
grayscale and color raster devices. Ghostscript 10.x renders aliased glyphs
that OCR frequently misreads as extra word breaks or substituted characters;
anti-aliasing materially improves OCR accuracy on the Ghostscript
rasterization path, especially for small fonts at moderate resolution. The
1-bit monochrome devices are unaffected, since they perform their own
anti-aliased downscaling and older Ghostscript versions reject alpha-bit
options on them. Note that the default rasterizer (--rasterizer auto)
prefers pypdfium2, which already anti-aliases; this change benefits users who
select --rasterizer ghostscript or do not have pypdfium2 installed.
OCRmyPDF now also logs which rasterizer rendered each page at debug verbosity
(-v 1), and the --rasterizer help text explains the OCR-quality
trade-off, to make such reports easier to diagnose. {issue}1439-v 1) so the original wording
is available for diagnosis. {issue}1566--mode strip, which removes the invisible OCR text layer from a PDF
in place. Unlike --ocr-engine none --force-ocr, it does not rasterize the
page, so images and visible content are preserved unchanged and the output is
smaller rather than larger. Only text drawn as invisible (PDF text render mode
3) is removed; some OCR engines -- and OCRmyPDF v2.2 and earlier -- express
text as visible glyphs covered by an opaque image, and that text cannot be
removed this way. {issue}1435end alias in --pages, denoting the last page
of the document. For example, --pages 3-end OCRs from page 3 through
the final page. {issue}1615--ghostscript-jpeg-quality and --ghostscript-jpeg-maxdpi
advanced options for tuning Ghostscript's PDF/A output. The optimizer's
--jpeg-quality remains the recommended file-size control.16851321TesseractConfigError with
actionable guidance, instead of crashing later with a confusing
FileNotFoundError on the missing hOCR output. {issue}1687_exec and subprocess modules to
separate probing from execution.PIL.Image.MAX_IMAGE_PIXELS
when the caller did not explicitly set max_image_mpixels. Host
applications (e.g. Paperless-NGX) that configure the PIL limit before
invoking ocrmypdf.ocr() now have their setting respected. The CLI
default of 250 megapixels is unchanged. {issue}166516661655work_folder not being set in PdfContext options when using
the Python API. Thanks @bluebox-steven. {issue}1613--no-overwrite / -n option to prevent overwriting output files.
If the destination file already exists, OCRmyPDF exits with code 5
(OutputFileAccessError). {issue}16421635optimize=2 or optimize=3 crash when using the Python API without
explicitly setting jpg_quality or png_quality. {issue}1641verapdf availability check crashing with NotADirectoryError on
some platforms. {issue}1638language parameter, always defaulting to
eng. The API now correctly maps language to OcrOptions languages
and splits +-separated codes (e.g. eng+deu) to match CLI behavior.
{issue}1640tesseract_timeout
defaulted to 0, causing Tesseract to time out immediately. The default is
now None, falling back to the plugin's 180-second timeout. {issue}16361630--image) for the hocrtransform tool,
enabling sandwich PDF output with the fpdf2 renderer. {issue}163416321631--redo-ocr
mode would shift text vertically on these files. {issue}16301612--tagged-pdf-mode to allow skipping the TaggedPDF error message, if desired.--mode force in particular).Breaking changes
OcrOptions objects instead of
argparse.Namespace objects. Most plugins will continue working due to duck-typing
compatibility, but plugin developers should update their type hints from Namespace
to OcrOptions.--jbig2-lossy and --jbig2-page-group-size options have been
removed due to well-documented risks of character substitution errors. These options are now
deprecated and will emit warnings if used. Only lossless JBIG2 compression is supported.--output-type auto (the new default) will produce a standard PDF instead of PDF/A. This is
a change from previous versions where Ghostscript was required and PDF/A was always produced.
This configuration is rare but users should be aware of the change.New features
pypdfium2 rasterizer: Added optional pypdfium2-based PDF rasterization plugin as an
alternative to Ghostscript for page rendering. Use --rasterizer pypdfium to enable
(requires pip install pypdfium2). The default --rasterizer auto prefers pypdfium when
available and falls back to Ghostscript.
Pluggable OCR engines: New --ocr-engine option allows selecting OCR engines:
auto (default): Uses Tesseracttesseract: Explicit Tesseract selectionnone: Skip OCR entirely for PDF processing-only workflowsThis prepares the foundation for future third-party OCR engine plugins.
Smart PDF/A conversion: New --output-type auto (now the default) produces best-effort
PDF/A output without requiring Ghostscript when the verapdf validator is available. Falls back
to traditional Ghostscript conversion when needed.
verapdf integration: Added optional verapdf validation for fast PDF/A conversion. When available, OCRmyPDF attempts speculative PDF/A conversion using pikepdf, validates with verapdf, and skips Ghostscript if validation passes.
Optional Ghostscript: As a consequence of the changes above, Ghostscript is no longer a required dependency. It is optional.
fpdf2 text renderer: Replaced legacy hOCR text renderer with new fpdf2-based implementation, providing better multilingual support and more accurate text positioning.
Improved Occulta glyphless font: The new Occulta font provides better handling of zero-width markers and double-width CJK characters for accurate text layer positioning.
Expanded multilingual font support: Added FontProvider infrastructure with language-aware font selection for Devanagari (Hindi, Sanskrit, Marathi, Nepali), CJK (Chinese, Japanese, Korean), Arabic script, and many other scripts. System font discovery reduces package size.
Simplified mode selection: New --mode (-m) argument consolidates processing options:
default: Error if text is found (standard behavior)force: Rasterize all content and run OCR (replaces --force-ocr)skip: Skip pages with existing text (replaces --skip-text)redo: Re-OCR pages, stripping old text layer (replaces --redo-ocr)Legacy flags remain as silent aliases for backward compatibility.
API improvements
OcrOptions Pydantic modelOcrElement, OcrClass, and BoundingBox exports for OCR engine plugin developersOcrEngine ABC with generate_ocr() method for direct OCR tree output, eliding the need to translate a modern engine's output to hOCR or directly write to PDF.Bug fixes
Documentation
--tesseract-timeout 0 with --ocr-engine none.Dependency changes
pypdfium2 or ghostscript for PDF rasterization (PDF to image)
verapdf or ghostscript for PDF/A generation
pypdfium2 for PDF rasterization (new dependency)ghostscript (used to be Required)verapdf for fast PDF/A validation (new dependency)fpdf2 for text layer rendering (new dependency)typer with cyclopts in misc scripts (new dependency)Migration guide for plugin developers
from ocrmypdf._options import OcrOptionsdef check_options(options: OcrOptions) instead of options: Namespaceoptions.languages, options.output_type, etc.