ContextQMD
Back to Photoprism

README

internal/ai/tensorflow/README.md

latest3.2 KB
Original Source

PhotoPrism — TensorFlow Package

Last Updated: March 3, 2026

Overview

internal/ai/tensorflow provides the shared TensorFlow helpers used by PhotoPrism’s built-in AI features (labels, NSFW, and FaceNet embeddings). It wraps SavedModel loading, input/output discovery, image tensor preparation, and label handling so higher-level packages can focus on domain logic.

Key Components

  • Model Loading — SavedModel, GetModelTagsInfo, and GetInputAndOutputFromSavedModel discover and load SavedModel graphs with appropriate tags.
  • Input Preparation — Image, ImageTransform, and ImageTensorBuilder convert JPEG images to tensors with the configured resolution, color order, and resize strategy.
  • Output Handling — AddSoftmax can insert a softmax op when a model exports logits.
  • Labels — LoadLabels loads label lists for classification models.

Model Loading Notes

  • Built-in models live under assets/models/ and are accessed via helpers in internal/ai/vision and internal/ai/classify.
  • When a model lacks explicit tags or signatures, the helpers attempt to infer input/output operations. Logs will show when inference kicks in.
  • Classification models may emit logits; if ModelInfo.Output.Logits is true, a softmax op is injected at load time.

Memory & Garbage Collection

TensorFlow tensors are allocated in C memory and freed by Go GC finalizers in the TensorFlow bindings. Long-running inference can therefore show increasing RSS even when the Go heap is small. PhotoPrism periodically triggers garbage collection to return freed C-allocated tensor buffers to the OS. Control this behavior with:

  • PHOTOPRISM_TF_GC_EVERY (default 200, 0 disables).
    Lower values reduce peak RSS but increase GC overhead and can slow indexing.

Troubleshooting Tips

  • Model fails to load: Verify the SavedModel path, tags, and that saved_model.pb plus variables/ exist under assets/models/<name>.
  • FaceNet load error Read less bytes than requested: The local assets/models/facenet/saved_model.pb file is usually incomplete or corrupted. Remove cached/downloaded files and reinstall:
    • rm -f /tmp/photoprism/facenet.zip
    • rm -rf assets/models/facenet
    • make dep-tensorflow (or scripts/download-facenet.sh)
    • Re-run the face tests (go test ./internal/ai/face -run TestNet -count=1)
  • Input/output mismatch: Check logs for inferred inputs/outputs and confirm vision.yml overrides (name, resolution, and TensorFlow.Input/Output).
  • Unexpected probabilities: Ensure logits are handled correctly and labels match output indices.
  • High memory usage: Confirm PHOTOPRISM_TF_GC_EVERY is set appropriately; model weights remain resident for the life of the process by design.