LIBERO

LIBERO is a benchmark designed to study lifelong robot learning — the idea that robots need to keep learning and adapting with their users over time, not just be pretrained once. It provides a set of standardized manipulation tasks that focus on knowledge transfer: how well a robot can apply what it has already learned to new situations. By evaluating on LIBERO, different algorithms can be compared fairly and researchers can build on each other's work.

• Paper: Benchmarking Knowledge Transfer for Lifelong Robot Learning
• GitHub: Lifelong-Robot-Learning/LIBERO
• Project website: libero-project.github.io

Available tasks

LIBERO includes five task suites covering 130 tasks, ranging from simple object manipulations to complex multi-step scenarios:

Installation

After following the LeRobot installation instructions:

pip install -e ".[libero]"

export MUJOCO_GL=egl  # for headless servers (HPC, cloud)

Evaluation

Default evaluation (recommended)

Evaluate across the four standard suites (10 episodes per task):

lerobot-eval \
  --policy.path="your-policy-id" \
  --env.type=libero \
  --env.task=libero_spatial,libero_object,libero_goal,libero_10 \
  --eval.batch_size=1 \
  --eval.n_episodes=10 \
  --env.max_parallel_tasks=1

Single-suite evaluation

Evaluate on one LIBERO suite:

lerobot-eval \
  --policy.path="your-policy-id" \
  --env.type=libero \
  --env.task=libero_object \
  --eval.batch_size=2 \
  --eval.n_episodes=3

• --env.task picks the suite (libero_object, libero_spatial, etc.).
• --env.task_ids restricts to specific task indices ([0], [1,2,3], etc.). Omit to run all tasks in the suite.
• --eval.batch_size controls how many environments run in parallel.
• --eval.n_episodes sets how many episodes to run per task.

Multi-suite evaluation

Benchmark a policy across multiple suites at once by passing a comma-separated list:

lerobot-eval \
  --policy.path="your-policy-id" \
  --env.type=libero \
  --env.task=libero_object,libero_spatial \
  --eval.batch_size=1 \
  --eval.n_episodes=2

Control mode

LIBERO supports two control modes — relative (default) and absolute. Different VLA checkpoints are trained with different action parameterizations, so make sure the mode matches your policy:

--env.control_mode=relative   # or "absolute"

Reset performance

By default, LeRobot preserves LIBERO's hard-reset behavior. With fixed initial states enabled, you can opt into soft resets to skip rebuilding the simulator model and renderer on every episode:

--env.init_states=true --env.hard_reset=false

Soft resets are faster but are not bit-identical to hard resets after the environment's settling steps, so camera observations and policy results may differ slightly. Use hard resets when reproducing benchmark results.

Policy inputs and outputs

Observations:

• observation.state — 8-dim proprioceptive features (eef position, axis-angle orientation, gripper qpos)
• observation.images.image — main camera view (agentview_image), HWC uint8
• observation.images.image2 — wrist camera view (robot0_eye_in_hand_image), HWC uint8

Actions:

• Continuous control in Box(-1, 1, shape=(7,)) — 6D end-effector delta + 1D gripper

Recommended evaluation episodes

For reproducible benchmarking, use 10 episodes per task across all four standard suites (Spatial, Object, Goal, Long). This gives 400 total episodes and matches the protocol used for published results. Success rates may vary by a few percent across evaluation seeds, so we recommend averaging over 3 seeds.

Training

Dataset

Two preprocessed LIBERO datasets are fully compatible with LeRobot. They contain the same demonstrations with the same schema and differ in how camera frames are stored:

We recommend lerobot/libero: 37× smaller download with equivalent loading speed (~330 samples/s per worker). Video re-encoding is slightly lossy; use the image-based variant if you cannot install a video decoding backend.

For reference, the original dataset published by Physical Intelligence:

• physical-intelligence/libero

Example training command

Train SmolVLA on the recommended dataset:

lerobot-train \
  --policy.type=smolvla \
  --policy.load_vlm_weights=true \
  --policy.push_to_hub=false \
  --dataset.repo_id=lerobot/libero \
  --dataset.video_backend=torchcodec \
  --output_dir=./outputs/libero_smolvla \
  --steps=100000 \
  --batch_size=64

To share the result on the Hub, replace --policy.push_to_hub=false with --policy.repo_id=${HF_USER}/libero-smolvla. Evaluate saved checkpoints with lerobot-eval as shown in the Evaluation section.

Reproducing published results

We reproduce the results of Pi0.5 on the LIBERO benchmark. We take the Physical Intelligence LIBERO base model (pi05_libero) and finetune for an additional 6k steps in bfloat16, with batch size of 256 on 8 H100 GPUs using the HuggingFace LIBERO dataset.

The finetuned model: lerobot/pi05_libero_finetuned

Evaluation command

lerobot-eval \
  --output_dir=./eval_logs/ \
  --env.type=libero \
  --env.task=libero_spatial,libero_object,libero_goal,libero_10 \
  --eval.batch_size=1 \
  --eval.n_episodes=10 \
  --policy.path=pi05_libero_finetuned \
  --policy.n_action_steps=10 \
  --env.max_parallel_tasks=1

We set n_action_steps=10, matching the original OpenPI implementation.

Results

These results are consistent with the original results reported by Physical Intelligence: