Vectorization and Throughput

Research snapshot: 2026-07-23. PufferLib's published 3.0.0 package and current 4.0 source line expose different vectorization systems. Never mix their configuration names.

Version split

The 4.0 package directory no longer contains the 3.0 vector.py, emulation.py, or pytorch.py modules. Treat old examples importing those modules as 3.0 examples, even if a stale copy remains under the 4.0 examples/ tree.

Published 3.0.0 API

The source signature is:

pufferlib.vector.make(
    env_creator_or_creators,
    env_args=None,
    env_kwargs=None,
    backend=pufferlib.PufferEnv,
    num_envs=1,
    seed=0,
    **kwargs,
)

Select a backend explicitly during development:

import pufferlib.vector

serial = pufferlib.vector.make(
    reviewed_env_creator,
    backend=pufferlib.vector.Serial,
    num_envs=4,
    seed=42,
)

parallel = pufferlib.vector.make(
    reviewed_env_creator,
    backend=pufferlib.vector.Multiprocessing,
    num_envs=16,
    num_workers=4,
    batch_size=8,
    zero_copy=True,
    seed=42,
)

Do not replace reviewed_env_creator with a dotted import string. Import the audited callable directly in trusted code. Environment construction executes package code and may initialize native libraries.

Stable backends

• PufferEnv is the default native backend. vector.make requires num_envs=1 for this backend because that one native environment can manage many agents internally.
• Serial runs multiple environment instances in the caller process. Use it for contract debugging and deterministic comparisons.
• Multiprocessing uses worker processes and shared arrays. It supports the synchronous reset/step facade and asynchronous async_reset/recv plus send/recv.
• Ray is an optional 3.0 backend and requires the package's pinned Ray extra. It introduces a separate distributed runtime and is not a safe local default.

Shape semantics

Do not assume num_envs == returned batch length.

• A single environment advertises num_agents, single_observation_space, and single_action_space.
• The full stable batch contains agent slots. For fixed-population environments, serial batch length is normally num_envs * num_agents.
• vecenv.agents_per_batch is the number of agent rows returned by one receive.
• Observations have leading agent-batch dimension; rewards, terminals, truncations, agent IDs, and masks have matching leading length.
• A synchronous call returns (observations, rewards, terminals, truncations, infos).
• The asynchronous recv() additionally returns agent_ids and masks.
• Structured observations/actions are flattened by emulation. Preserve the recorded dtype metadata and unflatten in the policy; do not cast arbitrary byte views to float first.

Validate actual shapes and dtypes at reset and the first step. In multi-agent workflows, use masks to exclude padded or inactive slots from loss and metrics.

Multiprocessing constraints

The 3.0 source validates these relationships:

1. num_envs must be divisible by num_workers.
2. batch_size defaults to num_envs.
3. batch_size must be divisible by num_envs / num_workers.
4. With zero-copy enabled, num_envs must be divisible by batch_size.
5. Physical-core oversubscription is rejected unless overwork=True; do not bypass this for benchmark headline numbers.

Always call close() in finally. Keep constructors top-level and serializable. Protect process creation with if __name__ == "__main__":.

Start methods

The stable API does not expose a start_method argument. The process context is therefore affected by Python and platform defaults. Set an application-wide method before constructing workers if your program requires one:

import multiprocessing as mp

if __name__ == "__main__":
    mp.set_start_method("spawn")
    main()

Prefer spawn when CUDA, threads, or non-fork-safe native libraries may already be initialized. Do not call set_start_method(..., force=True) inside a library. Record the effective method in benchmark output. forkserver can also be appropriate when available and tested. Never compare results that silently use different methods.

Seeding

• Pass one integer seed to vector.make; the stable implementation derives per-environment seeds.
• reset(seed=base_seed) similarly offsets seeds across serial environments.
• Seed action-space sampling separately when random actions are part of a test.
• A deterministic seed does not guarantee bitwise deterministic GPU training or deterministic third-party simulators.
• Recreate workers and environments for independent replicates; do not treat adjacent episodes from one long run as independent seeds.

Current 4.0 source

The current default branch uses native C environments and a different vector layout:

[vec]
total_agents = 4096
num_buffers = 2
num_threads = 16

Environment instances are grouped into buffers. Native execution uses OpenMP threads inside those buffers; rollout workers coordinate buffer transfers and, for GPU training, pinned memory and CUDA streams. The default trainer backend is native; --slowly selects the PyTorch fallback. Multi-GPU launch code explicitly uses a spawn multiprocessing context.

Build and test one audited Ocean environment at a time. The C binding defines observation size/type and action branches. A mismatch can cause memory corruption rather than a friendly Python shape error.

Benchmark methodology

Throughput is not a library constant. Report enough detail to reproduce it:

1. Pin source/package, Python, dependencies, compiler, and environment revision.
2. Record CPU model/core topology, GPU/driver/CUDA, OS, precision, backend, start method, env count, agent count, workers/threads, buffers, and batch.
3. State whether timing includes construction, reset, policy inference, host-device transfer, learning, rendering, logging, and checkpoint I/O.
4. Warm up separately; use a fixed number of agent steps, not only wall time.
5. Run at least three independent repeats; report all samples plus median and spread. Report failures and memory use.
6. Validate equivalent observations, actions, reset/autoreset behavior, frame skip, and policy workload before comparing backends.
7. Distinguish simulation SPS from end-to-end training SPS.

The 2024 compatibility paper benchmarked PufferLib 1.x-style vectorization on an i9-14900K/RTX 4090 desktop and an i7-10750H/RTX 3070 laptop. The 2025 PufferLib 2.0 paper reports a different Ocean/training system. Those results are scoped to their listed hardware and workloads; they are not expected values for 3.0 or 4.0.

Run the bundled bounded harness first:

python3 scripts/benchmark_vectorization.py --backend serial
python3 scripts/benchmark_vectorization.py \
  --backend multiprocessing --start-method spawn \
  --envs 8 --workers 2 --steps-per-env 2000

It benchmarks only the bundled synthetic environment, never imports PufferLib, and cannot substantiate an upstream PufferLib performance claim.

Sources

• PufferLib 3.0 vector source — stable API source; accessed 2026-07-23.
• PufferLib 3.0 vectorization example — stable usage example; accessed 2026-07-23.
• PufferLib 4.0 documentation — current native architecture and CLI; accessed 2026-07-23.
• PufferLib 4.0 trainer source — current config and spawn behavior; accessed 2026-07-23.
• PufferLib compatibility paper — submitted 2024-06-18.
• PufferLib 2.0 paper — Reinforcement Learning Journal, 2025.