ContextQMD
Back to Claude Scientific Skills

Environment Contracts and Native Environments

skills/pufferlib/references/environments.md

2.55.09.1 KB
Original Source

Environment Contracts and Native Environments

Research snapshot: 2026-07-23.

Start from the Gymnasium contract

A current single-agent Gymnasium environment defines observation_space and action_space, then implements:

python
def reset(self, *, seed=None, options=None):
    super().reset(seed=seed)
    return observation, info

def step(self, action):
    return observation, reward, terminated, truncated, info

Contract requirements:

  • observation must be contained in observation_space after reset and every step, with the documented shape and dtype.
  • action must be contained in action_space.
  • reward is a finite scalar for ordinary single-agent tasks.
  • terminated means the task's MDP reached a terminal state.
  • truncated means an external limit ended the episode, commonly a time limit.
  • info is a dictionary; never hide the only termination signal in it.
  • Call reset() after either terminated or truncated.
  • Seed the environment through reset(seed=...). Seed the action space separately when sampled actions must be reproducible.
  • Always call close().

Do not collapse terminated and truncated during learning. A time-limit truncation can still permit value bootstrapping; a true terminal state does not.

Run the local contract tool before involving PufferLib:

bash
python3 scripts/env_contract_validator.py

It validates only the bundled synthetic environment. It intentionally has no module-path option, so it cannot dynamically import an untrusted package.

Published PufferLib 3.0.0 native contract

For a native Python PufferEnv, assign these attributes before calling super().__init__(buf):

python
import gymnasium
import numpy as np
import pufferlib


class ReviewedEnv(pufferlib.PufferEnv):
    def __init__(self, buf=None, seed=0):
        self.single_observation_space = gymnasium.spaces.Box(
            low=-1.0, high=1.0, shape=(4,), dtype=np.float32
        )
        self.single_action_space = gymnasium.spaces.Discrete(3)
        self.num_agents = 2
        super().__init__(buf)

The stable base accepts a Box observation space and Discrete, MultiDiscrete, or Box action space. It allocates or attaches:

  • observations
  • actions
  • rewards
  • terminals
  • truncations
  • masks

Native methods operate on those buffers:

python
def reset(self, seed=None):
    # update self.observations in place
    return self.observations, []

def step(self, actions):
    # update all buffers in place
    return (
        self.observations,
        self.rewards,
        self.terminals,
        self.truncations,
        [],
    )

The infos value for native Puffer environments is a list of dictionaries. PufferLib's native interface expects vector rows for agents, even when there is one agent. Native environments handle their own resets; clear rewards, terminals, truncations, masks, and partially written observations explicitly. Never leave a previous step's buffer values in place.

Native shape checklist

For A = num_agents and single observation shape S:

  • observations: (A, *S)
  • rewards: (A,)
  • terminals: (A,)
  • truncations: (A,)
  • masks: (A,)
  • actions: joint shape derived from the single action space and A

Validate the exact allocated action shape rather than assuming (A,), especially for MultiDiscrete and Box actions.

Stable Gymnasium and PettingZoo adaptation

PufferLib 3.0 uses explicit adapters:

python
import pufferlib.emulation

wrapped = pufferlib.emulation.GymnasiumPufferEnv(reviewed_gymnasium_instance)

or:

python
wrapped = pufferlib.emulation.PettingZooPufferEnv(reviewed_parallel_instance)

There is no supported 3.0 pufferlib.emulate(...) convenience function matching the old skill examples. Pass either an env instance or an env_creator callable according to the class signature; do not pass both.

The Gymnasium adapter:

  • maps structured observation/action spaces to flat arrays;
  • checks the first observation and action against the original spaces;
  • returns separate terminal and truncation values;
  • requires reset before step and reset after episode end.

The PettingZoo adapter:

  • targets the Parallel API;
  • uses possible_agents as the fixed slot set;
  • pads missing agents and exposes masks;
  • canonicalizes per-agent spaces and flattened buffers.

Validate heterogeneous-agent spaces before use. The adapter derives its single spaces from the first possible agent, so environments with incompatible spaces need an explicit reviewed transformation.

Structured spaces

Stable emulation supports Box, Discrete, MultiDiscrete, Tuple, and Dict patterns through a packed NumPy dtype. This is byte-layout conversion, not semantic feature engineering. Check:

  • deterministic Dict key order;
  • leaf shape and dtype;
  • finite numeric values;
  • lossless action reconstruction;
  • policy-side unflattening;
  • padding/mask handling for variable populations.

Current 4.0 Ocean contract

The 4.0 default branch focuses on first-party C environments. It no longer provides the 3.0 Python emulation/vector modules. The official starting points are:

  • ocean/squared: commented single-agent template
  • ocean/target: commented multi-agent template

A binding defines compile-time metadata such as:

c
#define OBS_SIZE 121
#define NUM_ATNS 1
#define ACT_SIZES {5}
#define OBS_TENSOR_T ByteTensor

#define Env Squared
#include "vecenv.h"

The environment struct must include pointers for observations, actions, rewards, and terminals, plus num_agents and a log struct. It implements c_reset, c_step, c_render, and c_close; binding.c supplies my_init and my_log.

Security and correctness rules:

  1. Treat the C environment and every linked library as native code.
  2. Verify repository/commit, license, asset rights, and checksums before build.
  3. Build only the selected environment in a disposable container or VM.
  4. Start with the local/address-sanitizer build described by upstream.
  5. Match OBS_SIZE, tensor dtype, action branch count/sizes, and actual writes.
  6. Bounds-check every index and allocation; use checked arithmetic for sizes.
  7. Initialize every output element each step. Reset reward/terminal buffers before early returns.
  8. Use an environment-owned RNG seeded per instance; do not use global RNG state for reproducibility.
  9. Free only memory owned by the environment. Do not free framework buffers.
  10. Fuzz reset/step/action boundaries before optimization.

c_step may reset immediately after marking a terminal. Record this autoreset behavior when interpreting terminal observations.

Environment provenance

An environment package may execute arbitrary Python/native code and may fetch assets at import, build, reset, or render time. Before execution:

  • use the official repository and immutable revision;
  • inspect package/build scripts and transitive dependencies;
  • verify artifact hashes or attestations;
  • review license compatibility for code, datasets, media, ROMs, maps, and model opponents separately;
  • reject unlicensed ROMs or “accept ROM license” automation without proof of rights;
  • disable network and credentials in the first-run sandbox;
  • cap disk, memory, processes, threads, episode length, agents, and render size;
  • do not load bundled checkpoints or pickle files during environment import.

An entry in Ocean/config is not a blanket security, quality, or licensing approval.

Testing ladder

  1. Built-in synthetic contract validator.
  2. One environment, one seed, serial, tens of steps.
  3. Boundary actions and intentionally invalid actions.
  4. Termination and time-limit truncation tests.
  5. Same-seed trace comparison.
  6. Independent-seed diversity check.
  7. Structured-space round trip.
  8. Multi-agent join/leave and mask tests.
  9. Serial versus vectorized trace equivalence where ordering permits.
  10. Bounded throughput benchmark only after correctness passes.

Sources