optional (see below)

Models

Structural conditioning uses canny edge or depth detection to maintain precise control during image transformations. By preserving the original image's structure through edge or depth maps, users can make text-guided edits while keeping the core composition intact. This is particularly effective for retexturing images. We release four variations: two based on edge maps (full model and LoRA for FLUX.1 [dev]) and two based on depth maps (full model and LoRA for FLUX.1 [dev]).

Examples

Open-weights usage

The weights will be downloaded automatically to checkpoints/ from HuggingFace once you start one of the demos. Alternatively, you may download the weights manually and put them in checkpoints/, or you can also manually link them with the following environment variables:

export FLUX_MODEL=<your model path here>
export FLUX_AE=<your autoencoder path here>

# optional (see below)
export FLUX_LORA=<your lora path here>

Note that the LoRA models (flux-dev-canny-lora and flux-dev-depth-lora) require the base FLUX.1 [dev] model to be downloaded first. The system will automatically download both the base model and the LoRA adapter when using these variants.

For interactive sampling run

python -m flux control --name <name> --loop

where name is one of flux-dev-canny, flux-dev-depth, flux-dev-canny-lora, or flux-dev-depth-lora.

TRT engine inference

We provide exports in BF16, FP8, and FP4 precision. Note that you need to install the repository with TensorRT support as outlined here.

python flux control --name=<name> --loop --img_cond_path="assets/robot.webp" --trt --static_shape=False --trt_transformer_precision <precision>

where <precision> is either bf16, fp8, or fp4.

Diffusers usage

Flux Control (including the LoRAs) is also compatible with the diffusers Python library. Check out the documentation to learn more.