waypoint CLI reference

Targets waypoint-bio 1.0.2 (PyPI) / 1.0.4 (GitHub main, commit f45eee6, 2026-07-16).

waypoint {pretrain,benchmark,finetune,embed,prepare-dataset} ...

Config paths are resolved first against the working directory, then against the bundled waypoint_bio/configs/ tree inside the installed wheel. So --config configs/benchmark.yaml works from anywhere without cloning the repo. The same fallback applies to the bundled example data (examples/abundance_matrix.tsv, examples/finetune_classification.parquet, …).

waypoint prepare-dataset

Converts a sample × taxa abundance matrix into waypoint format.

auto treats the file as taxa-as-rows when the first column header is taxonomy, lineage, taxon, otu, or #otu id (case-insensitive); otherwise samples-as-rows with the first column as the sample ID.

--taxonomy_format genus prefixes bare column names with g__. It disables higher-rank fallback, because a bare name carries no lineage to fall back to — prefer real lineage strings.

waypoint embed

One fixed-size vector per sample from a pretrained checkpoint. No fine-tuning, no labels needed.

Output columns are dim_0 … dim_{H-1}, indexed by sample ID. Hidden size H is 256 (6m), 512 (45m), 768 (170m).

Behaviour worth knowing: tokens that map to <unk> are dropped before ordering, not encoded. A row with no in-vocabulary taxa still produces an output row, but its sequence is [BOS][EOS] and the embedding is meaningless. Run scripts/vocab_coverage.py first.

Ordering: by descending abundance z-score when token_std_means.parquet is present (it ships with every published checkpoint and with waypoint pretrain output), otherwise by descending raw relative abundance.

waypoint finetune

Fine-tunes a checkpoint on your own labelled waypoint-format data.

Fine-tuning config keys

split_column: null        # column holding train/validation/test; null = random split
val_fraction: 0.1
test_fraction: 0.1

max_length: 512           # must match the checkpoint's pretraining context
pooling_strategy: last_token
filter_unk_taxa: true     # drop out-of-vocabulary taxa rather than feed <unk>

seed: 42
learning_rate: 0.00003
num_epochs: 1             # raise this; early stopping is what should terminate training
batch_size: 64
warmup_steps: 1000        # lower to ~50 for small datasets
weight_decay: 0.001
eval_strategy: steps
eval_steps: 400
logging_steps: 5
patience: 5               # eval steps without improvement before early stopping
save_total_limit: 1

use_lora: false
lora_r: 8
lora_alpha: 16            # convention: 2 * r
lora_dropout: 0.05
lora_target_modules: [c_attn, c_proj]   # GPT-2 fused QKV and output projection
lora_bias: none
lora_fan_in_fan_out: true               # required for GPT-2 Conv1D layouts

num_epochs: 1 in the shipped configs is tuned for the large Compass tasks. On a few-thousand-row dataset one epoch is a handful of optimizer steps and the model barely moves — raise num_epochs and let patience stop it. Likewise eval_steps: 400 may never fire; lower it so early stopping and best-checkpoint selection can actually work.

LoRA adapters are merged back into the base transformer before saving, so best_model/ loads with a plain AutoModel.from_pretrained and works with waypoint embed and waypoint benchmark.

Outputs

waypoint benchmark

configs/benchmark.yaml is the fine-tuning config applied identically to every task: learning_rate: 3e-5, num_epochs: 1, batch_size: 64, warmup_steps: 1000, weight_decay: 0.001, patience: 5, pooling_strategy: last_token, eval_steps: 400, filter_unk_taxa: true, seed: 42. Change it and your score is no longer comparable to the paper.

The paper reports means over three independent runs. A single run is noisy; vary --seed and report the spread.

waypoint pretrain

Steps: download the Atlas pretrain split → build a taxonomic tokenizer from the corpus → compute per-token abundance mean/std for z-score ordering → train GPT-2 with next-token prediction and early stopping → save best_model/.

configs/pretraining.yaml

training_type: next_token_prediction
taxon_rank: genus              # tokenization rank; changing it means re-pretraining
fallback_to_higher_rank: true  # use the most specific higher rank when genus is absent
max_length: 512
learning_rate: 0.001
warmup_steps: 1000
weight_decay: 0.001
batch_size: 32
num_epochs: 100
patience: 10
eval_steps: 3261
save_steps: 3261
logging_steps: 100
val_split: 0.1
seed: 42

Architectures

All share model_type: gpt2, n_positions: 512, and a fixed per-head dimension of 64.

Only 6m, 45m, and 170m are published as checkpoints. The rest exist so the paper's scaling study is reproducible; gpt2-6m-mgm isolates the effect of head count against the MGM baseline.

Parameter counts exclude token and positional embeddings, so the Hub's reported sizes are larger (the 6m checkpoint reports ~10.1M, the 45m ~51.8M).

Pretraining Atlas end to end is a multi-GPU-day job. Validate the pipeline with --max_samples 5000 before committing to a full run.